-- Logs begin at Sat 2022-10-15 12:10:06 UTC, end at Sat 2022-10-15 13:13:38 UTC. -- Oct 15 12:42:18.942912 dv-cinder-861491-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 15 12:42:18.951840 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Oct 15 12:42:18 2022] *** Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: nodename: dv-cinder-861491-2-iscsi Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: machine: x86_64 Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: clock source: unix Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: pcre jit disabled Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: detected number of CPU cores: 4 Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: current working directory: / Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: detected binary path: /usr/bin/uwsgi-core Oct 15 12:42:18.955689 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: your processes number limit is 47680 Oct 15 12:42:18.956289 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: your memory page size is 4096 bytes Oct 15 12:42:18.956289 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: detected max file descriptor number: 2048 Oct 15 12:42:18.956289 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: lock engine: pthread robust mutexes Oct 15 12:42:18.956289 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: thunder lock: enabled Oct 15 12:42:18.956289 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 15 12:42:18.956289 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 15 12:42:18.996589 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: Python main interpreter initialized at 0x557bd7f799c0 Oct 15 12:42:18.996589 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: python threads support enabled Oct 15 12:42:18.996589 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: your server socket listen backlog is limited to 100 connections Oct 15 12:42:18.996589 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: your mercy for graceful operations on workers is 90 seconds Oct 15 12:42:18.996926 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: mapped 403077 bytes (393 KB) for 2 cores Oct 15 12:42:18.997023 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: *** Operational MODE: preforking *** Oct 15 12:42:18.997319 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: *** uWSGI is running in multiple interpreter mode *** Oct 15 12:42:18.997319 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: spawned uWSGI master process (pid: 81598) Oct 15 12:42:18.997525 dv-cinder-861491-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 15 12:42:18.998501 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: spawned uWSGI worker 1 (pid: 81612, cores: 1) Oct 15 12:42:18.998501 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: spawned uWSGI worker 2 (pid: 81613, cores: 1) Oct 15 12:42:18.998652 dv-cinder-861491-2-iscsi devstack@keystone.service[81598]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 15 12:42:23.431074 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 12:42:23.431074 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: from cryptography.utils import int_from_bytes Oct 15 12:42:23.431881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 12:42:23.431881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: from cryptography.utils import int_from_bytes Oct 15 12:42:23.433083 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 12:42:23.433083 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: from cryptography.utils import int_from_bytes Oct 15 12:42:23.433578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 12:42:23.433578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: from cryptography.utils import int_from_bytes Oct 15 12:42:23.482976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 12:42:23.483176 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 12:42:23.483369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] command line args: [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 12:42:23.483642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 12:42:23.483822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ================================================================================ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 12:42:23.484079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] admin_token = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.484298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 12:42:23.484382 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config_dir = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.484498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 12:42:23.484589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.484666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config_source = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.484735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] command line args: [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 12:42:23.484840 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] debug = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.484958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 12:42:23.485042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.485305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ================================================================================ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 12:42:23.485426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] admin_token = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.485530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.485674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.485773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_dir = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.485877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.485987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.485987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_source = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.486173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.486278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] debug = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.486378 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.486485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.486603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_config_append = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.486702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.486800 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.486900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_dir = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487190 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487290 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487405 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487679 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_config_append = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487823 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.487969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_dir = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.488089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.488172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.488242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.488345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.488468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.488600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.488682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.488854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.488935 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.489086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.489241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.489335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.489407 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.489559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.489644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.489758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.489831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.489952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.490032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.490155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.490289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.490395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.490634 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.490777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.490861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.490960 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] publish_errors = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.491111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.491229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.491362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.491444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.491567 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.491645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.491812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] standard_threads = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] publish_errors = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492913 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.492989 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.493088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_journal = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.493164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.493289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_json = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.493420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.493553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_stderr = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.493634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_syslog = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.493711 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] standard_threads = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.493843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.493925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.494097 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.494185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.494278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.494359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.494459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_journal = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.494579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.494660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_json = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.494806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_stderr = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.494883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.494953 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_syslog = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.495066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.495184 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 12:42:23.495266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.495443 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.495520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.495684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.495773 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.495946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.496022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.496188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.496270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.496433 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.496507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.496666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.496749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.496912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.496988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.497145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.497240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.497421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.497504 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.497665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.497749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.497901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.497987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.498721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.498812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.498985 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.499066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.499230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.499311 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.499465 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.499548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.499721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.499797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.499963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.500044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.500225 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.500312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.500471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.500551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.500700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.500776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.500931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.501080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.501153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.501253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.501411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.501489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.501657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.501744 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.501904 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.501980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.502133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.502209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.502363 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.502447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.502608 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.502684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.502846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.502927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.503079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.503165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.503328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.503404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.503556 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.503632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.503790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.503877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.504025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.504107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.504253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.504334 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.504481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.504596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.504692 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.504786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.504924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.505059 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.505167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.505313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.505425 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.505525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.505642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.505786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.505896 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.505993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.506086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.506188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.506316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.506425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.506551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.506660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.506797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.506907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.507026 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.507133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.507262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.507370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.507495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.507595 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.507727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.507827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508123 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508323 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508641 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508874 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.508977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.509110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.509243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.509366 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.509486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.509597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.509692 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.509826 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.509929 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.510064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.510164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.510300 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.510448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.510528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.510598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.510759 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.510844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.511012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.511089 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.511254 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.511330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.511491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.511568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.511723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.511799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.511957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.512032 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.512191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.512269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.512439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.512519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.512676 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.512752 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.512912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.512988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.513155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.513289 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.513417 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.513495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.513699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.514342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.514488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.514664 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.514742 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.515000 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.515081 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.515251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.515326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.515491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.515568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.515730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.515846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.515977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.516053 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.516216 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.516294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.516459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.516536 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.516697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.516773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.516936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.517011 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.517178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.517272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.517440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.517520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.517678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.517792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.517912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.517989 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.518142 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.518219 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.518374 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.518452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.518603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.518686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.518836 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.518913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.519064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.519147 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.519304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.519380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.519539 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.519615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.519770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.519846 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.520009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.520084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.520244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.520319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.520488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.520566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.520730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.520813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.520959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.external = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.521033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.521212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.521292 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.521486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.521575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.521724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.521801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.521955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.password = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.522032 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.522185 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.token = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.522264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.external = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.522421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.522498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.522655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.522731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.522901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.522979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.523131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.523208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.password = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.523376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.523454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.token = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.523623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.523699 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.523851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.523927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.524085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.524170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.524322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.524399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.524566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.524649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.524815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.524900 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.525052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.525129 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.525318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.525399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.525551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.525635 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.525798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.525914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.526051 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.526127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.526285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.526369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.526536 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.526612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.526776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.526858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.527026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.527174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.527283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.527361 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.527525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.527603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.527780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.527864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.528016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.528094 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.528254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.528331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.528505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.528589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.528752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.528830 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.528981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.529058 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.529248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.529375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.529543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.529620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.529821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.529995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.530067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.530146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.530268 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.530347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.530520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.530598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.530763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.530850 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.531017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.531102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.531257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.531334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.531501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.531590 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.531736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.531820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.531970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.532096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.532209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.532294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.532436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.532513 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.532673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.532750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.533175 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.533343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.533443 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.533538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.535981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.536116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.536216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.536322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.536421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.536516 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.536611 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.536706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.536809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.536906 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537305 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.537902 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538004 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538271 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.538994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539202 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539791 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.539932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540288 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540570 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540793 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.540931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541000 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541148 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541148 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.541895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.542069 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.542185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.542319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.542455 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.542558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.542687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.542794 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.542927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.543033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.543159 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.543264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.543409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.543507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.543664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.543751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.543908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.543985 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.544150 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.544245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.544393 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.544478 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.544630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.544707 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.544876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.544998 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.545177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.545376 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.545499 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.545610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.545697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.545771 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.545886 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.546057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.546179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.546305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.546431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.546556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.546687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.546813 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.546940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.547064 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.547064 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.547299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.547434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.547557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.547692 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.547815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.547940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.548065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.548065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.548306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.548429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.548553 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.548677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.548801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.548933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549059 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549424 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549791 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.549860 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.550035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.550116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.550272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.550349 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.550507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.550584 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.550743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.550822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.550991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.551070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.551232 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.551309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.551462 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.551541 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.551699 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.551777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.551937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.552051 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.552181 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.552286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.552426 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.552529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.552663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.552767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.552897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.553000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.553139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.553268 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.553399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.553503 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.553646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.553751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.553881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.553985 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.554118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.554294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.554413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.554512 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.554607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.554702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.554835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.554937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.555041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.555139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.555268 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.555381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.555515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.555619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.555752 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.555855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.556023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.556146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.556262 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.556924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557745 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557893 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.557966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.558037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.558142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.558299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.558385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.558536 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.558614 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.558786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.558863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.559035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.559125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.559282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.559360 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.559529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.559735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.559812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.559976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.560054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.560232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.560309 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.560469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.561072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.561303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.561386 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.561542 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.561624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.561777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.561993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.562080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.562260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.562338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.562506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.562583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.562744 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.562829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.562979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.563056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.563209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.563286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.563443 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.563528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.563680 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.563759 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.563926 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.564004 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.564161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.564245 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.564414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.564492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.564652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.564729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.564886 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.564970 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.565123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.565213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.565393 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.565471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.565637 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.565725 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.565874 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.565952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.566111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.566188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.566343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.566425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.566586 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.566664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.566839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.566915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.567074 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.567160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.567316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.567397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.567555 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.567631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.567799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.567876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.568038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.568117 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.568290 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.568367 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.568527 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.568604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.568772 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.568851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.569006 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.569093 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.569264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.569348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.569501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.569655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.569730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.569801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.569969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.570048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.570205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.570290 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.570448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.570526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.570681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.570758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.570927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.571013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.571162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.571240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.571406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.571482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.571634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.571719 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.571873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.571951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.572107 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.572184 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.572277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 12:42:23.572363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.572591 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.572833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.573086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.573347 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.573595 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.573842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 12:42:23.574016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 12:42:23.602018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.602246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.602470 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.603207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.603447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.603677 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.611628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.612716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.639501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.639727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.640834 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.641075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.693809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.694288 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.742109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.742338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.743530 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.743774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.898360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.898590 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.898678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.898797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.898881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.899075 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.899166 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.899242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.899313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.899418 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.899500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.899662 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.922984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.923220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.923376 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.923467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.923591 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.923693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.923821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.923904 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.924049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.924132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.924263 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.924346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.924479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.924698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.935323 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.935838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.948136 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.948354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.948449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.948526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.948598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.948751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.948834 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.948970 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.949051 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.949210 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.949306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.949440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.949560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.949646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 12:42:23.966774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.967091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 12:42:23.967511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.967599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.967678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 12:42:23.967753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 12:42:23.967878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.967974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.968079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 12:42:23.968165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.968353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.968437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.968616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.968709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.968878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.969134 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.978265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.978550 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.978647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.978826 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.978910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.979114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.979198 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.979413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.986698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:23.987020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:23.989626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:23.989909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:23.989998 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.990195 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.990283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.990479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.990562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.990762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.990854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.991027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.991109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.991296 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.991376 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.991568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.991650 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.991839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.991919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.992109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.992189 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.992378 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.992459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:23.992643 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.017454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.018799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.018922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.019179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.020284 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.020579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.023814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.024103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.024385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.025285 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.025577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.025863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.031297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.031581 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.033035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.033369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.037513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.037809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.038098 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.038387 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.038668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.038946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.039114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.039208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.039417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.039500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.039715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.040014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.040300 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.040579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.040853 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.041126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.055753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.056034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.059523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.059807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.059936 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.063651 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.063956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.064228 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.067455 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.067760 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.068089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.071161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.071931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.074332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.074632 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.074917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.075037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.075203 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.075483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.075764 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.078296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.078604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.078897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.079188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.079476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.079762 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.091528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.092366 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.092739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.093005 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.096108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.096402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.096677 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.096948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.099810 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.101368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.102879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.103157 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.103828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.105402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.105965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.106251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.106541 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.106825 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.106945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.107211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.107292 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.110268 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.110561 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.110849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.111136 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.111537 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.118364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.118647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.118927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.119212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.119591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.119879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.120159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.122730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.123018 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.123884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.124171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.124555 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.124839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.125120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.136044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.139041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.140645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.141936 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.143481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.143653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.145085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.145405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.145812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.146582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.148050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.149543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.149834 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.150246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.152088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.153684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.157436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.157531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.158098 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.158385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.158663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.159169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.162102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.162508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.162800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.163078 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.166420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.166714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.166996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.171169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.171465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.171753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.174058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.174343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.174623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.174913 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.175187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.175458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.175731 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.176015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.176283 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.176555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.176825 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.177099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.177388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.177647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.178651 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 12:42:24.178950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.179237 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.179518 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.179794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.180068 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.180347 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.180634 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.180916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.181197 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.181495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.181771 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.182042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.182325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 12:42:24.211914 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557bd7f799c0 pid: 81613 (default app) Oct 15 12:42:24.215126 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1fa3-646f-4aab-9987-063229d14c78 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:24.215411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1fa3-646f-4aab-9987-063229d14c78 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:24.215682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1fa3-646f-4aab-9987-063229d14c78 None None] PATH_INFO: `/v3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:24.217770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557bd7f799c0 pid: 81612 (default app) Oct 15 12:42:24.220351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Sat Oct 15 12:42:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:32.011281 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5301767-6c0c-4662-9758-2a5b8736655b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:32.011765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5301767-6c0c-4662-9758-2a5b8736655b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:32.011765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5301767-6c0c-4662-9758-2a5b8736655b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:32.016832 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:42:32.022181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:32.022668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:32.022764 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:32.101915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: INFO dbcounter [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] Registered counter for database keystone Oct 15 12:42:32.125290 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG oslo_db.sqlalchemy.engines [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81613) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 12:42:32.131431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writer thread running {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 12:42:32.310539 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG passlib.handlers.bcrypt [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81613) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 15 12:42:32.311102 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG passlib.handlers.bcrypt [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81613) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 15 12:42:32.369685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:32.371353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:32.501792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 15 12:42:32.941201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5301767-6c0c-4662-9758-2a5b8736655b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:33.010987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: INFO dbcounter [None req-a5301767-6c0c-4662-9758-2a5b8736655b None None] Registered counter for database keystone Oct 15 12:42:33.029908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG oslo_db.sqlalchemy.engines [None req-a5301767-6c0c-4662-9758-2a5b8736655b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81612) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 12:42:33.036392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writer thread running {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 12:42:33.154581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5301767-6c0c-4662-9758-2a5b8736655b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:33.159275 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f262552-bba2-4942-a8ed-546413d33128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:33.159895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f262552-bba2-4942-a8ed-546413d33128 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:33.160176 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f262552-bba2-4942-a8ed-546413d33128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:33.160450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f262552-bba2-4942-a8ed-546413d33128 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:33.168522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f262552-bba2-4942-a8ed-546413d33128 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:33.219990 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0f262552-bba2-4942-a8ed-546413d33128 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 15 12:42:33.220278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f262552-bba2-4942-a8ed-546413d33128 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:33.229517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Sat Oct 15 12:42:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 12:42:33.236018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:33.252630 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f5edb-dee0-463a-98c8-f99aac05fb24 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:33.257056 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a1515-d5bb-4713-b7b9-616516924806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:33.258175 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a1515-d5bb-4713-b7b9-616516924806 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:33.258463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a1515-d5bb-4713-b7b9-616516924806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:33.258740 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a1515-d5bb-4713-b7b9-616516924806 admin admin] PATH_INFO: `/v3/projects/90479a5430a046cea76f80fced5060d9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:33.260319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a1515-d5bb-4713-b7b9-616516924806 admin admin] RBAC: Authorizing `identity:get_project(project_id=90479a5430a046cea76f80fced5060d9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:33.313780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING keystone.common.rbac_enforcer.enforcer [None req-640a1515-d5bb-4713-b7b9-616516924806 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 15 12:42:33.314130 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a1515-d5bb-4713-b7b9-616516924806 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:33.318681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 12:42:33 2022] GET /identity/v3/projects/90479a5430a046cea76f80fced5060d9 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:35.448476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd5067-368d-4c01-b472-4c7146e1bc0e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:35.449096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd5067-368d-4c01-b472-4c7146e1bc0e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:35.449096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd5067-368d-4c01-b472-4c7146e1bc0e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:35.453330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:35.458860 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d9855-d279-4aae-8500-d13da2181f69 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:35.459252 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d9855-d279-4aae-8500-d13da2181f69 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:35.459420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d9855-d279-4aae-8500-d13da2181f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:35.481687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-821d9855-d279-4aae-8500-d13da2181f69 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:35.483869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-821d9855-d279-4aae-8500-d13da2181f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:35.492692 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 12:42:35.803345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd5067-368d-4c01-b472-4c7146e1bc0e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:35.819947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd5067-368d-4c01-b472-4c7146e1bc0e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:35.825112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b44857-5f67-4963-b61b-ae1c42e2a44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:35.826297 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b44857-5f67-4963-b61b-ae1c42e2a44e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:35.826569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b44857-5f67-4963-b61b-ae1c42e2a44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:35.826858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b44857-5f67-4963-b61b-ae1c42e2a44e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:35.835096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b44857-5f67-4963-b61b-ae1c42e2a44e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:35.836893 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b44857-5f67-4963-b61b-ae1c42e2a44e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:35.847819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Sat Oct 15 12:42:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 12:42:35.854123 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d9855-d279-4aae-8500-d13da2181f69 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:35.863330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d9855-d279-4aae-8500-d13da2181f69 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:35.867839 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01d67e-badb-4786-9a51-b0954a34efb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:35.868670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01d67e-badb-4786-9a51-b0954a34efb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:35.868951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01d67e-badb-4786-9a51-b0954a34efb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:35.869253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01d67e-badb-4786-9a51-b0954a34efb4 admin admin] PATH_INFO: `/v3/users/05701e65a5f34ab3837cd0f450bded1e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:35.871033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01d67e-badb-4786-9a51-b0954a34efb4 admin admin] RBAC: Authorizing `identity:get_user(user_id=05701e65a5f34ab3837cd0f450bded1e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:35.872983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01d67e-badb-4786-9a51-b0954a34efb4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:35.874632 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Sat Oct 15 12:42:35 2022] GET /identity/v3/users/05701e65a5f34ab3837cd0f450bded1e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:38.093406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539cf94-6095-48eb-b69e-9af7d9652b16 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:38.093945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539cf94-6095-48eb-b69e-9af7d9652b16 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:38.093945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539cf94-6095-48eb-b69e-9af7d9652b16 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:38.095284 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:38.100449 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30892986-10b6-43b6-ab88-54a3d0bc73be None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:38.100878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30892986-10b6-43b6-ab88-54a3d0bc73be None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:38.101131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30892986-10b6-43b6-ab88-54a3d0bc73be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:38.120793 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-30892986-10b6-43b6-ab88-54a3d0bc73be None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:38.122188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-30892986-10b6-43b6-ab88-54a3d0bc73be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:38.163868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059096aa-99d9-4e7c-aa82-212ea5aae509 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:38.164151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059096aa-99d9-4e7c-aa82-212ea5aae509 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:38.164428 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059096aa-99d9-4e7c-aa82-212ea5aae509 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:38.165791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:42:38.171596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f7a79-d832-4c2c-933a-201dc2ac74d5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:38.171876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f7a79-d832-4c2c-933a-201dc2ac74d5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:38.172149 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f7a79-d832-4c2c-933a-201dc2ac74d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:38.207173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:38.214147 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG passlib.handlers.bcrypt [None req-176f7a79-d832-4c2c-933a-201dc2ac74d5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81612) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 15 12:42:38.214703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG passlib.handlers.bcrypt [None req-176f7a79-d832-4c2c-933a-201dc2ac74d5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81612) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 15 12:42:38.264370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-176f7a79-d832-4c2c-933a-201dc2ac74d5 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:38.265967 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-176f7a79-d832-4c2c-933a-201dc2ac74d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:38.364947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:42:38.476611 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30892986-10b6-43b6-ab88-54a3d0bc73be None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:38.502326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:38.582928 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30892986-10b6-43b6-ab88-54a3d0bc73be None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:38.615891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aac46-11f1-4a8b-8ef9-c841574f6ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:38.616467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aac46-11f1-4a8b-8ef9-c841574f6ddc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:38.616752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aac46-11f1-4a8b-8ef9-c841574f6ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:38.617032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aac46-11f1-4a8b-8ef9-c841574f6ddc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:38.628818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12aac46-11f1-4a8b-8ef9-c841574f6ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:42:38.634157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f7a79-d832-4c2c-933a-201dc2ac74d5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a12aac46-11f1-4a8b-8ef9-c841574f6ddc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:42:38.637855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:38.639500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:42:38.640490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:42:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:42:38.647141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aac46-11f1-4a8b-8ef9-c841574f6ddc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:38.729518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:38.734412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f7a79-d832-4c2c-933a-201dc2ac74d5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:38.736884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:38.749068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aac46-11f1-4a8b-8ef9-c841574f6ddc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:38.771876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:38.772913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:38.773500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:38.773788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:38.774765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:38.776206 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:38.782952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec316c15-4fee-44ce-9a8f-c136d295d4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:38.783818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec316c15-4fee-44ce-9a8f-c136d295d4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:38.784108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec316c15-4fee-44ce-9a8f-c136d295d4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:38.784384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec316c15-4fee-44ce-9a8f-c136d295d4b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:38.785050 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec316c15-4fee-44ce-9a8f-c136d295d4b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:38.785393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec316c15-4fee-44ce-9a8f-c136d295d4b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:38.786780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec316c15-4fee-44ce-9a8f-c136d295d4b7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:38.799111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:42:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:42:38.804742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec316c15-4fee-44ce-9a8f-c136d295d4b7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:38.840521 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.sql.core [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 12:42:38.840521 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 12:42:38.840521 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [parameters: {'id': '0641f98b009646a4842749797b3272d0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 12:42:38.840521 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81612) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:42:38.860081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 12:42:38.860886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application yield resource Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise self.value Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application session.commit() Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 12:42:38.862173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.flush() Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application compat.raise_( Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 12:42:38.863328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 12:42:38.864266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:42:38.865016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [parameters: {'id': '0641f98b009646a4842749797b3272d0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 12:42:38.866772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:38.867751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 8/16] 10.222.0.30 () {60 vars in 1137 bytes} [Sat Oct 15 12:42:38 2022] POST /identity/v3/domains => generated 190 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:42:38.868816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:38.906141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec316c15-4fee-44ce-9a8f-c136d295d4b7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:38.940107 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b54a7-f400-44df-bcfc-c2ef67c9d66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:38.940821 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b54a7-f400-44df-bcfc-c2ef67c9d66c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:38.941170 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b54a7-f400-44df-bcfc-c2ef67c9d66c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:38.941487 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b54a7-f400-44df-bcfc-c2ef67c9d66c None admin] PATH_INFO: `/v3/users/05701e65a5f34ab3837cd0f450bded1e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:38.942620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b54a7-f400-44df-bcfc-c2ef67c9d66c None admin] RBAC: Authorizing `identity:get_user(user_id=05701e65a5f34ab3837cd0f450bded1e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:38.944070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b54a7-f400-44df-bcfc-c2ef67c9d66c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:38.945819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 9/17] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:42:38 2022] GET /identity/v3/users/05701e65a5f34ab3837cd0f450bded1e => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:42:38.951573 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b54a7-f400-44df-bcfc-c2ef67c9d66c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:38.965911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:38.972868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808fadb-368e-4949-85cc-b98f93dbf33c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:39.005688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a30d10-c105-425b-8b31-bb2634d3f9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:39.006538 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a30d10-c105-425b-8b31-bb2634d3f9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:39.006815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a30d10-c105-425b-8b31-bb2634d3f9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:39.007089 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a30d10-c105-425b-8b31-bb2634d3f9a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:39.025392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a30d10-c105-425b-8b31-bb2634d3f9a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:39.026879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a30d10-c105-425b-8b31-bb2634d3f9a7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e2a30d10-c105-425b-8b31-bb2634d3f9a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:42:39.043747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:39.044723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:42:39.045582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:42:39.045582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:42:39.045582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:39.045582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 9/18] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:42:39.051062 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a30d10-c105-425b-8b31-bb2634d3f9a7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:39.055134 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b54a7-f400-44df-bcfc-c2ef67c9d66c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:39.073072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:39.091239 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa136e-a357-453e-abb3-03b11c5b7f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:39.092383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa136e-a357-453e-abb3-03b11c5b7f15 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:39.092634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa136e-a357-453e-abb3-03b11c5b7f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:39.092910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa136e-a357-453e-abb3-03b11c5b7f15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:39.094067 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa136e-a357-453e-abb3-03b11c5b7f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:39.095518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa136e-a357-453e-abb3-03b11c5b7f15 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:39.097099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 10/19] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:42:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:42:39.102785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa136e-a357-453e-abb3-03b11c5b7f15 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:39.151910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a30d10-c105-425b-8b31-bb2634d3f9a7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:39.182892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:39.189164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a9ac4-511f-4d98-9c26-9b05ded20801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:39.189854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a9ac4-511f-4d98-9c26-9b05ded20801 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:39.190135 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a9ac4-511f-4d98-9c26-9b05ded20801 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:39.190412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a9ac4-511f-4d98-9c26-9b05ded20801 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:39.191057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a9ac4-511f-4d98-9c26-9b05ded20801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:39.191364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a9ac4-511f-4d98-9c26-9b05ded20801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:39.192748 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a9ac4-511f-4d98-9c26-9b05ded20801 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:39.202932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa136e-a357-453e-abb3-03b11c5b7f15 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:39.210230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 10/20] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:42:39.237665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b33146-8eb7-471a-82d3-569ae2c26761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:39.238361 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b33146-8eb7-471a-82d3-569ae2c26761 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:39.238645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b33146-8eb7-471a-82d3-569ae2c26761 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:39.238926 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b33146-8eb7-471a-82d3-569ae2c26761 None admin] PATH_INFO: `/v3/domains/default/users/05701e65a5f34ab3837cd0f450bded1e/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:39.240974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b33146-8eb7-471a-82d3-569ae2c26761 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=05701e65a5f34ab3837cd0f450bded1e, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:39.242528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b33146-8eb7-471a-82d3-569ae2c26761 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:39.413009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 11/21] 10.222.0.30 () {58 vars in 1361 bytes} [Sat Oct 15 12:42:39 2022] PUT /identity/v3/domains/default/users/05701e65a5f34ab3837cd0f450bded1e/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 12:42:41.974075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c246c1-6461-4fe9-ade4-eb49d7ac576d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:41.974579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c246c1-6461-4fe9-ade4-eb49d7ac576d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:41.974579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c246c1-6461-4fe9-ade4-eb49d7ac576d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:41.975892 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 11/22] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:41.983111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c830a4-24bc-4707-adf2-6a4b99b75e8f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:41.983614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c830a4-24bc-4707-adf2-6a4b99b75e8f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:41.983684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c830a4-24bc-4707-adf2-6a4b99b75e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.012848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-86c830a4-24bc-4707-adf2-6a4b99b75e8f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:42.015186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-86c830a4-24bc-4707-adf2-6a4b99b75e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:42.061401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5705bcf-6c17-4b7b-b6e0-3996ac5ff5a9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:42.061699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5705bcf-6c17-4b7b-b6e0-3996ac5ff5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:42.061970 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5705bcf-6c17-4b7b-b6e0-3996ac5ff5a9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.063316 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 12/23] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:42.077881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76796a8-0e90-4bf7-ac4b-76a85858dd42 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:42.078477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76796a8-0e90-4bf7-ac4b-76a85858dd42 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:42.078966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76796a8-0e90-4bf7-ac4b-76a85858dd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.097145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 12/24] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:42.098457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faafa4b-38b4-41d0-8463-af77b3ebbb7c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:42.098747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faafa4b-38b4-41d0-8463-af77b3ebbb7c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:42.099025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faafa4b-38b4-41d0-8463-af77b3ebbb7c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.103400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 13/25] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:42.113236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe7220-7a6a-4aab-b182-93bed1862b1a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:42.113532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe7220-7a6a-4aab-b182-93bed1862b1a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:42.113818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe7220-7a6a-4aab-b182-93bed1862b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.116643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a76796a8-0e90-4bf7-ac4b-76a85858dd42 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:42.118602 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a76796a8-0e90-4bf7-ac4b-76a85858dd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:42.150328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e3fe7220-7a6a-4aab-b182-93bed1862b1a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:42.151668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e3fe7220-7a6a-4aab-b182-93bed1862b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:42.241376 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 13/26] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:42:42.242630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e89306-7696-4438-b1f7-2e399b2aa00a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:42.242920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e89306-7696-4438-b1f7-2e399b2aa00a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:42.243199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e89306-7696-4438-b1f7-2e399b2aa00a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.244757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 14/27] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:42.257153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de3967-ab41-452c-913d-7fb4e787e854 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:42.257473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de3967-ab41-452c-913d-7fb4e787e854 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:42.257752 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de3967-ab41-452c-913d-7fb4e787e854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.273282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:42.287398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 14/28] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:42:42.289179 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a6de3967-ab41-452c-913d-7fb4e787e854 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:42.290411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a6de3967-ab41-452c-913d-7fb4e787e854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:42.293540 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:42.380227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 15/29] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:42:42.454472 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe7220-7a6a-4aab-b182-93bed1862b1a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:42.595486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe7220-7a6a-4aab-b182-93bed1862b1a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:42.636630 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed463a81-5a3f-467b-bd3e-7d51069d8ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:42.637358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed463a81-5a3f-467b-bd3e-7d51069d8ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:42.637649 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed463a81-5a3f-467b-bd3e-7d51069d8ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:42.637928 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed463a81-5a3f-467b-bd3e-7d51069d8ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.649438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed463a81-5a3f-467b-bd3e-7d51069d8ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:42.651050 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed463a81-5a3f-467b-bd3e-7d51069d8ce2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ed463a81-5a3f-467b-bd3e-7d51069d8ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:42:42.677737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:42.678628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:42:42.679423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:42:42.679423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:42:42.679423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:42:42.679423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 15/30] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:42:42.684706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de3967-ab41-452c-913d-7fb4e787e854 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:42.775609 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed463a81-5a3f-467b-bd3e-7d51069d8ce2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:42.801886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de3967-ab41-452c-913d-7fb4e787e854 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:42.837122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3f264-2691-41ef-93e7-584551786eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:42.837560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:42.837747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3f264-2691-41ef-93e7-584551786eba None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:42.837933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3f264-2691-41ef-93e7-584551786eba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:42.838212 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3f264-2691-41ef-93e7-584551786eba None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.838862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3f264-2691-41ef-93e7-584551786eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:42.839167 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3f264-2691-41ef-93e7-584551786eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:42.840576 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3f264-2691-41ef-93e7-584551786eba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:42.849767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:42.853562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 16/31] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:42.858858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3f264-2691-41ef-93e7-584551786eba None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:42.901974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed463a81-5a3f-467b-bd3e-7d51069d8ce2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:42.952024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc68c8-7d75-45cb-8496-f19e0cd8a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:42.953287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc68c8-7d75-45cb-8496-f19e0cd8a30b None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:42.953773 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc68c8-7d75-45cb-8496-f19e0cd8a30b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:42.954491 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc68c8-7d75-45cb-8496-f19e0cd8a30b None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:42.955780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc68c8-7d75-45cb-8496-f19e0cd8a30b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:42.957428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc68c8-7d75-45cb-8496-f19e0cd8a30b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:42.971162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3f264-2691-41ef-93e7-584551786eba None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:43.007094 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e0183-36d2-46f7-8bef-6165ffeae7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:43.007740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e0183-36d2-46f7-8bef-6165ffeae7ca None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:43.008002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e0183-36d2-46f7-8bef-6165ffeae7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:43.008398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e0183-36d2-46f7-8bef-6165ffeae7ca None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:43.009018 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e0183-36d2-46f7-8bef-6165ffeae7ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:43.010469 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e0183-36d2-46f7-8bef-6165ffeae7ca None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:43.064912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 16/32] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 12:42:42 2022] POST /identity/v3/roles => generated 219 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:42:43.071268 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc68c8-7d75-45cb-8496-f19e0cd8a30b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:43.249697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc68c8-7d75-45cb-8496-f19e0cd8a30b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:43.265927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:43.272205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 17/33] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 12:42:42 2022] POST /identity/v3/roles => generated 225 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:42:43.280324 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e0183-36d2-46f7-8bef-6165ffeae7ca None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:43.293486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d216267-960f-4498-ab98-75e6e3db7be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:43.294145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d216267-960f-4498-ab98-75e6e3db7be0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:43.294595 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d216267-960f-4498-ab98-75e6e3db7be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:43.295452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d216267-960f-4498-ab98-75e6e3db7be0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:43.311809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d216267-960f-4498-ab98-75e6e3db7be0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2d216267-960f-4498-ab98-75e6e3db7be0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:42:43.318441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:43.319321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:42:43.320080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 17/34] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:42:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:42:43.321994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af7de-cee3-4a6e-ab46-b0046ad7d5ea None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:43.322277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af7de-cee3-4a6e-ab46-b0046ad7d5ea None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:43.322554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af7de-cee3-4a6e-ab46-b0046ad7d5ea None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:43.324343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 18/35] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:43.331160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976ba8-42a4-4f5f-9f42-0e44cc08236b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:43.331953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976ba8-42a4-4f5f-9f42-0e44cc08236b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:43.337951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976ba8-42a4-4f5f-9f42-0e44cc08236b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:43.358923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d6976ba8-42a4-4f5f-9f42-0e44cc08236b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:43.360531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d6976ba8-42a4-4f5f-9f42-0e44cc08236b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:43.381627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:43.431734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e0183-36d2-46f7-8bef-6165ffeae7ca None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:43.474986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94209d90-93d0-4ba1-bad9-fed8935c1e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:43.475652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94209d90-93d0-4ba1-bad9-fed8935c1e04 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:43.475935 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94209d90-93d0-4ba1-bad9-fed8935c1e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:43.476220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94209d90-93d0-4ba1-bad9-fed8935c1e04 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:43.476899 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94209d90-93d0-4ba1-bad9-fed8935c1e04 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:43.478609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94209d90-93d0-4ba1-bad9-fed8935c1e04 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:43.563773 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 19/36] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:42:43.564648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976ba8-42a4-4f5f-9f42-0e44cc08236b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:43.594961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 18/37] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:42:43 2022] POST /identity/v3/projects => generated 301 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:42:43.694613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976ba8-42a4-4f5f-9f42-0e44cc08236b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:43.697021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:43.738035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039701d-2a51-4084-9807-7f374d74f716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:43.739053 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039701d-2a51-4084-9807-7f374d74f716 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:43.739560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039701d-2a51-4084-9807-7f374d74f716 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:43.740039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039701d-2a51-4084-9807-7f374d74f716 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:43.740949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039701d-2a51-4084-9807-7f374d74f716 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:43.741483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039701d-2a51-4084-9807-7f374d74f716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:43.743137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039701d-2a51-4084-9807-7f374d74f716 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:43.754763 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 20/38] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:42:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:43.760196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94209d90-93d0-4ba1-bad9-fed8935c1e04 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:43.813676 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496eb09-b0fd-4bf0-97a3-9c23da45448b None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:43.814275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496eb09-b0fd-4bf0-97a3-9c23da45448b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:43.814776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496eb09-b0fd-4bf0-97a3-9c23da45448b None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:43.816291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 21/39] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:43.821326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ccaed-ad9a-4045-9da2-0364c0849514 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:43.821623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ccaed-ad9a-4045-9da2-0364c0849514 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:43.821904 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ccaed-ad9a-4045-9da2-0364c0849514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:43.844056 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ba8ccaed-ad9a-4045-9da2-0364c0849514 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:43.845300 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ba8ccaed-ad9a-4045-9da2-0364c0849514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:43.881934 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94209d90-93d0-4ba1-bad9-fed8935c1e04 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:43.896411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:43.916877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46634c-c583-4341-951a-1847af2d1ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:43.917553 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46634c-c583-4341-951a-1847af2d1ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:43.917926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46634c-c583-4341-951a-1847af2d1ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:43.918349 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46634c-c583-4341-951a-1847af2d1ba6 None admin] PATH_INFO: `/v3/users/05701e65a5f34ab3837cd0f450bded1e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:43.918448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 22/40] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:42:43.919103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ccaed-ad9a-4045-9da2-0364c0849514 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:43.922429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46634c-c583-4341-951a-1847af2d1ba6 None admin] RBAC: Authorizing `identity:get_user(user_id=05701e65a5f34ab3837cd0f450bded1e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:43.924714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46634c-c583-4341-951a-1847af2d1ba6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:43.929329 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 19/41] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:42:43 2022] GET /identity/v3/users/05701e65a5f34ab3837cd0f450bded1e => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:43.932237 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46634c-c583-4341-951a-1847af2d1ba6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:44.024606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:44.031619 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ccaed-ad9a-4045-9da2-0364c0849514 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:44.039951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46634c-c583-4341-951a-1847af2d1ba6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:44.067772 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee3df8-81d8-4109-9889-da0cb1387d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:44.068320 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee3df8-81d8-4109-9889-da0cb1387d6a None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:44.068601 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee3df8-81d8-4109-9889-da0cb1387d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:44.068875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee3df8-81d8-4109-9889-da0cb1387d6a None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:44.069575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee3df8-81d8-4109-9889-da0cb1387d6a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:44.070951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee3df8-81d8-4109-9889-da0cb1387d6a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:44.075957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2884a-ad33-43b1-b605-829d3a7e2dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:44.076593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2884a-ad33-43b1-b605-829d3a7e2dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:44.076877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2884a-ad33-43b1-b605-829d3a7e2dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:44.077156 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2884a-ad33-43b1-b605-829d3a7e2dd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:44.078331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2884a-ad33-43b1-b605-829d3a7e2dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:44.079776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2884a-ad33-43b1-b605-829d3a7e2dd8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:44.081370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 20/42] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:42:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:42:44.086929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2884a-ad33-43b1-b605-829d3a7e2dd8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:44.137108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:44.139796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 23/43] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 12:42:43 2022] POST /identity/v3/roles => generated 223 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:42:44.196219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee3df8-81d8-4109-9889-da0cb1387d6a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:44.204205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2884a-ad33-43b1-b605-829d3a7e2dd8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:44.246234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0763950-8aa7-41cc-a5f4-3e2d34f0cc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:44.248168 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0763950-8aa7-41cc-a5f4-3e2d34f0cc70 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:44.249423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0763950-8aa7-41cc-a5f4-3e2d34f0cc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:44.249724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0763950-8aa7-41cc-a5f4-3e2d34f0cc70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:44.250485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0763950-8aa7-41cc-a5f4-3e2d34f0cc70 None admin] RBAC: Adding query filter params (role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.domain.id=default, user.id=05701e65a5f34ab3837cd0f450bded1e) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:44.250805 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0763950-8aa7-41cc-a5f4-3e2d34f0cc70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:44.252341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0763950-8aa7-41cc-a5f4-3e2d34f0cc70 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:44.264241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 21/44] 10.222.0.30 () {56 vars in 1322 bytes} [Sat Oct 15 12:42:44 2022] GET /identity/v3/role_assignments?user.id=05701e65a5f34ab3837cd0f450bded1e&scope.domain.id=default&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 530 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:42:44.324784 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee3df8-81d8-4109-9889-da0cb1387d6a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:44.366587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d6975-feda-4464-add9-cbd3719b5cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:44.367245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d6975-feda-4464-add9-cbd3719b5cea None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:44.367531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d6975-feda-4464-add9-cbd3719b5cea None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:44.367808 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d6975-feda-4464-add9-cbd3719b5cea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:44.368973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d6975-feda-4464-add9-cbd3719b5cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:44.370499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d6975-feda-4464-add9-cbd3719b5cea None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:44.372167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 24/45] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:42:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:42:44.378004 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0763950-8aa7-41cc-a5f4-3e2d34f0cc70 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:44.485144 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0763950-8aa7-41cc-a5f4-3e2d34f0cc70 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:44.498613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:44.519223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d3c89-b305-4b0e-929d-4474a950c80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:44.519836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d3c89-b305-4b0e-929d-4474a950c80e None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:44.520118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d3c89-b305-4b0e-929d-4474a950c80e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:44.520396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d3c89-b305-4b0e-929d-4474a950c80e None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:44.521060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d3c89-b305-4b0e-929d-4474a950c80e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:44.522567 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d3c89-b305-4b0e-929d-4474a950c80e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:44.595709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 22/46] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:42:44 2022] POST /identity/v3/projects => generated 312 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:42:46.728086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bda52-e480-4caf-831f-f798400b6e6f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:46.728621 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bda52-e480-4caf-831f-f798400b6e6f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:46.728621 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bda52-e480-4caf-831f-f798400b6e6f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:46.730073 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 25/47] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:46.736152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b89a78-ff08-4b20-8192-7d0b886b85b9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:46.736564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b89a78-ff08-4b20-8192-7d0b886b85b9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:46.736705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b89a78-ff08-4b20-8192-7d0b886b85b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:46.754652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b5b89a78-ff08-4b20-8192-7d0b886b85b9 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:46.756160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b5b89a78-ff08-4b20-8192-7d0b886b85b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:46.820716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 23/48] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:42:47.089977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bda52-e480-4caf-831f-f798400b6e6f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:47.114383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:47.189278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bda52-e480-4caf-831f-f798400b6e6f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:47.222630 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea7991-8cd0-44a7-9da4-8b9e8340fb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:47.223253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea7991-8cd0-44a7-9da4-8b9e8340fb8e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:47.223535 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea7991-8cd0-44a7-9da4-8b9e8340fb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:47.223813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea7991-8cd0-44a7-9da4-8b9e8340fb8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:47.224915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea7991-8cd0-44a7-9da4-8b9e8340fb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:47.226389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea7991-8cd0-44a7-9da4-8b9e8340fb8e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:47.227940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 26/49] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:42:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:47.233636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b89a78-ff08-4b20-8192-7d0b886b85b9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:47.323230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:47.330257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b89a78-ff08-4b20-8192-7d0b886b85b9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:47.363220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f3291-3de5-486b-aa88-afde9c2b8dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:47.363765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f3291-3de5-486b-aa88-afde9c2b8dd2 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:47.364043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f3291-3de5-486b-aa88-afde9c2b8dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:47.364318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f3291-3de5-486b-aa88-afde9c2b8dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:47.364972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f3291-3de5-486b-aa88-afde9c2b8dd2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:47.366596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f3291-3de5-486b-aa88-afde9c2b8dd2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:47.475447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 24/50] 10.222.0.30 () {60 vars in 1140 bytes} [Sat Oct 15 12:42:47 2022] POST /identity/v3/projects => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:42:49.663002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec8e4e-6fb3-415f-8918-8021175cafdc None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:49.663507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec8e4e-6fb3-415f-8918-8021175cafdc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:49.663590 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec8e4e-6fb3-415f-8918-8021175cafdc None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:49.664907 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 27/51] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:49.670130 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf48748-e5bb-47f5-bd55-61e3a0f0545b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:49.670516 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf48748-e5bb-47f5-bd55-61e3a0f0545b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:49.670694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf48748-e5bb-47f5-bd55-61e3a0f0545b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:49.689373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7bf48748-e5bb-47f5-bd55-61e3a0f0545b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:49.690572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7bf48748-e5bb-47f5-bd55-61e3a0f0545b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:49.755266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 25/52] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:50.028924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec8e4e-6fb3-415f-8918-8021175cafdc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:50.119833 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:50.130468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec8e4e-6fb3-415f-8918-8021175cafdc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:50.165793 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0009e6-c770-4767-9f1f-77520e4f21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:50.167064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0009e6-c770-4767-9f1f-77520e4f21fa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:50.167924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0009e6-c770-4767-9f1f-77520e4f21fa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:50.168800 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0009e6-c770-4767-9f1f-77520e4f21fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:50.170836 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0009e6-c770-4767-9f1f-77520e4f21fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:50.172318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0009e6-c770-4767-9f1f-77520e4f21fa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:50.173959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 28/53] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:42:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:50.179508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf48748-e5bb-47f5-bd55-61e3a0f0545b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:50.260679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:50.289655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf48748-e5bb-47f5-bd55-61e3a0f0545b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:50.333182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2732d-cf3e-4691-9849-164f9364fdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:50.334057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2732d-cf3e-4691-9849-164f9364fdf2 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:50.334454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2732d-cf3e-4691-9849-164f9364fdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:50.334843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2732d-cf3e-4691-9849-164f9364fdf2 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:50.335782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2732d-cf3e-4691-9849-164f9364fdf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:50.337933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2732d-cf3e-4691-9849-164f9364fdf2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:50.452008 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 26/54] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:42:50 2022] POST /identity/v3/users => generated 275 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:42:53.848821 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd617b-7398-40fe-a92a-1ba6682c7a48 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:53.849912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd617b-7398-40fe-a92a-1ba6682c7a48 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:53.850407 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd617b-7398-40fe-a92a-1ba6682c7a48 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:53.852027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 29/55] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:53.858450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa657a9-5e4d-485b-9259-04c416dd332c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:53.859322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa657a9-5e4d-485b-9259-04c416dd332c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:53.859805 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa657a9-5e4d-485b-9259-04c416dd332c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.014277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bfa657a9-5e4d-485b-9259-04c416dd332c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:54.015864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bfa657a9-5e4d-485b-9259-04c416dd332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:54.053316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957ee3b-9243-48d0-8c40-ab2af0ad879e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.053986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957ee3b-9243-48d0-8c40-ab2af0ad879e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.054480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957ee3b-9243-48d0-8c40-ab2af0ad879e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.056199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 30/56] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:54.062421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaba218-eb61-46aa-9e3e-c57813420653 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.063109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaba218-eb61-46aa-9e3e-c57813420653 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.063605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaba218-eb61-46aa-9e3e-c57813420653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.094205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4eaba218-eb61-46aa-9e3e-c57813420653 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:54.095462 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4eaba218-eb61-46aa-9e3e-c57813420653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:54.147550 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 27/57] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:54.164522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b7fb6-eef6-43ed-adeb-6faf789cbbfd None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.165151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b7fb6-eef6-43ed-adeb-6faf789cbbfd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.165656 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b7fb6-eef6-43ed-adeb-6faf789cbbfd None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.167351 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 28/58] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:42:54.177145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52821789-8a1d-4dca-92c1-ddc6319bf8e5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.178050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52821789-8a1d-4dca-92c1-ddc6319bf8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.178660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52821789-8a1d-4dca-92c1-ddc6319bf8e5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.180405 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 29/59] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:54.181702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa5f58-b252-441d-9c2f-08c521351d0f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.182211 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa5f58-b252-441d-9c2f-08c521351d0f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.187875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa5f58-b252-441d-9c2f-08c521351d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.191480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 31/60] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:42:54.192749 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cecaf6-a0c9-4bdc-b925-19dc0bf73f38 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.193040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cecaf6-a0c9-4bdc-b925-19dc0bf73f38 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.197602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cecaf6-a0c9-4bdc-b925-19dc0bf73f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.209018 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a9fa5f58-b252-441d-9c2f-08c521351d0f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:54.210528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a9fa5f58-b252-441d-9c2f-08c521351d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:54.235755 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b5cecaf6-a0c9-4bdc-b925-19dc0bf73f38 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:54.236976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b5cecaf6-a0c9-4bdc-b925-19dc0bf73f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:54.292947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:54.314469 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 30/61] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:54.366253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 32/62] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:54.436541 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa5f58-b252-441d-9c2f-08c521351d0f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:54.492578 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cecaf6-a0c9-4bdc-b925-19dc0bf73f38 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:54.518824 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:54.630172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa5f58-b252-441d-9c2f-08c521351d0f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:54.636034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cecaf6-a0c9-4bdc-b925-19dc0bf73f38 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:54.667958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4a688-39c2-4c24-9ebe-357e6a4b74a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:54.668611 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4a688-39c2-4c24-9ebe-357e6a4b74a0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.668892 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4a688-39c2-4c24-9ebe-357e6a4b74a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.669904 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4a688-39c2-4c24-9ebe-357e6a4b74a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.679742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88ccd4-ecac-4210-9665-4a65e52b6ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:54.680719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88ccd4-ecac-4210-9665-4a65e52b6ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.681502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88ccd4-ecac-4210-9665-4a65e52b6ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.681781 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4a688-39c2-4c24-9ebe-357e6a4b74a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:42:54.682215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88ccd4-ecac-4210-9665-4a65e52b6ab2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-52e4a688-39c2-4c24-9ebe-357e6a4b74a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:42:54.688464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:54.689460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:54.690257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 31/63] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:42:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:42:54.690961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4a688-39c2-4c24-9ebe-357e6a4b74a0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:54.698568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88ccd4-ecac-4210-9665-4a65e52b6ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ee88ccd4-ecac-4210-9665-4a65e52b6ab2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:42:54.707658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:54.708516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:42:54.709302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:42:54.710486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 33/64] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:42:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:42:54.712692 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88ccd4-ecac-4210-9665-4a65e52b6ab2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:54.760017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:54.802992 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:54.820249 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4a688-39c2-4c24-9ebe-357e6a4b74a0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:54.825293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88ccd4-ecac-4210-9665-4a65e52b6ab2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:54.860550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d610c1-252c-4fa7-9a31-3a8e4240c9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:54.861229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecb8fc-cbbc-4ddb-8d8d-446269754f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:54.861595 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d610c1-252c-4fa7-9a31-3a8e4240c9af None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.861695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecb8fc-cbbc-4ddb-8d8d-446269754f0a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:54.861793 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d610c1-252c-4fa7-9a31-3a8e4240c9af None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.861885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecb8fc-cbbc-4ddb-8d8d-446269754f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:54.861969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d610c1-252c-4fa7-9a31-3a8e4240c9af None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.862053 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecb8fc-cbbc-4ddb-8d8d-446269754f0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:54.862420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d610c1-252c-4fa7-9a31-3a8e4240c9af None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:54.862683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d610c1-252c-4fa7-9a31-3a8e4240c9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:54.864076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d610c1-252c-4fa7-9a31-3a8e4240c9af None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:54.871984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 34/65] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:42:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:54.872745 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ecb8fc-cbbc-4ddb-8d8d-446269754f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:42:54.877346 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d610c1-252c-4fa7-9a31-3a8e4240c9af None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c4ecb8fc-cbbc-4ddb-8d8d-446269754f0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:42:54.877791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:54.878660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:54.879417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 32/66] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:42:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:42:54.880246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecb8fc-cbbc-4ddb-8d8d-446269754f0a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:54.978468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d610c1-252c-4fa7-9a31-3a8e4240c9af None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:54.978667 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecb8fc-cbbc-4ddb-8d8d-446269754f0a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:54.991046 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.012186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dee622-bb69-4c74-92ef-c5a496f63390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.013470 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dee622-bb69-4c74-92ef-c5a496f63390 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.013917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18291598-faf6-4625-874e-d2994d5c548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.014051 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dee622-bb69-4c74-92ef-c5a496f63390 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.014051 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dee622-bb69-4c74-92ef-c5a496f63390 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.014204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18291598-faf6-4625-874e-d2994d5c548c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.014461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18291598-faf6-4625-874e-d2994d5c548c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.014725 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dee622-bb69-4c74-92ef-c5a496f63390 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:55.014831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18291598-faf6-4625-874e-d2994d5c548c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.015042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dee622-bb69-4c74-92ef-c5a496f63390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.016431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dee622-bb69-4c74-92ef-c5a496f63390 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.024596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 35/67] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:42:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.025679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18291598-faf6-4625-874e-d2994d5c548c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:42:55.026461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101be40-4b37-4e14-895d-5bc0b32a1e85 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.026759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101be40-4b37-4e14-895d-5bc0b32a1e85 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.027046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101be40-4b37-4e14-895d-5bc0b32a1e85 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.029177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 36/68] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-18291598-faf6-4625-874e-d2994d5c548c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:42:55.031511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:55.032478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:55.033631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 33/69] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:42:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:42:55.034276 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18291598-faf6-4625-874e-d2994d5c548c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.034348 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101be40-4b37-4e14-895d-5bc0b32a1e85 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.037562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.132758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18291598-faf6-4625-874e-d2994d5c548c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.136629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101be40-4b37-4e14-895d-5bc0b32a1e85 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.166605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d155001-4d9a-4a9b-9008-0c227a40e2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.167207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d155001-4d9a-4a9b-9008-0c227a40e2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.167487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d155001-4d9a-4a9b-9008-0c227a40e2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.167758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d155001-4d9a-4a9b-9008-0c227a40e2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.168417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d155001-4d9a-4a9b-9008-0c227a40e2f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:55.168721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d155001-4d9a-4a9b-9008-0c227a40e2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.170187 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d155001-4d9a-4a9b-9008-0c227a40e2f6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.170282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a99eb4-9331-484c-bc54-3c9834cdc232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.170872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a99eb4-9331-484c-bc54-3c9834cdc232 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.171159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a99eb4-9331-484c-bc54-3c9834cdc232 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.171445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a99eb4-9331-484c-bc54-3c9834cdc232 None admin] PATH_INFO: `/v3/users/98eb25bc1113448eb680b306b130079c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.181792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 34/70] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.182604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d155001-4d9a-4a9b-9008-0c227a40e2f6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.186370 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a99eb4-9331-484c-bc54-3c9834cdc232 None admin] RBAC: Authorizing `identity:get_user(user_id=98eb25bc1113448eb680b306b130079c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.188611 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a99eb4-9331-484c-bc54-3c9834cdc232 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.190319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 37/71] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/users/98eb25bc1113448eb680b306b130079c => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.191464 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6abd5-f671-458d-b2ea-83890a2b46fb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.192183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6abd5-f671-458d-b2ea-83890a2b46fb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.192467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6abd5-f671-458d-b2ea-83890a2b46fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.210717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e7a6abd5-f671-458d-b2ea-83890a2b46fb None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:55.211919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e7a6abd5-f671-458d-b2ea-83890a2b46fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:55.218046 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.267906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.279201 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 38/72] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:55.282044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6abd5-f671-458d-b2ea-83890a2b46fb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.285686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d155001-4d9a-4a9b-9008-0c227a40e2f6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.319588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6867c-72f2-414b-a84c-3b33ab9205dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.320231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6867c-72f2-414b-a84c-3b33ab9205dd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.320581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6867c-72f2-414b-a84c-3b33ab9205dd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.320860 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6867c-72f2-414b-a84c-3b33ab9205dd None admin] PATH_INFO: `/v3/users/05701e65a5f34ab3837cd0f450bded1e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.322038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6867c-72f2-414b-a84c-3b33ab9205dd None admin] RBAC: Authorizing `identity:get_user(user_id=05701e65a5f34ab3837cd0f450bded1e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.323528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6867c-72f2-414b-a84c-3b33ab9205dd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.330190 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 35/73] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/users/05701e65a5f34ab3837cd0f450bded1e => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.330190 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6867c-72f2-414b-a84c-3b33ab9205dd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.387032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6abd5-f671-458d-b2ea-83890a2b46fb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.428766 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11536f2-c278-46cf-88e7-ef26ee5abd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.429837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11536f2-c278-46cf-88e7-ef26ee5abd05 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.430258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11536f2-c278-46cf-88e7-ef26ee5abd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.430675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11536f2-c278-46cf-88e7-ef26ee5abd05 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.431350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11536f2-c278-46cf-88e7-ef26ee5abd05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:55.431816 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11536f2-c278-46cf-88e7-ef26ee5abd05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.433844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11536f2-c278-46cf-88e7-ef26ee5abd05 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.435145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6867c-72f2-414b-a84c-3b33ab9205dd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.436680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.448281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 39/74] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.448281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11536f2-c278-46cf-88e7-ef26ee5abd05 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.473418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e430c-ba42-4b79-ac5e-aeccd80f5eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.474104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e430c-ba42-4b79-ac5e-aeccd80f5eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.474385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e430c-ba42-4b79-ac5e-aeccd80f5eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.474898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e430c-ba42-4b79-ac5e-aeccd80f5eb3 None admin] PATH_INFO: `/v3/users/98eb25bc1113448eb680b306b130079c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.476890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e430c-ba42-4b79-ac5e-aeccd80f5eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=98eb25bc1113448eb680b306b130079c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.478477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e430c-ba42-4b79-ac5e-aeccd80f5eb3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.484538 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 36/75] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/users/98eb25bc1113448eb680b306b130079c => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.484538 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e430c-ba42-4b79-ac5e-aeccd80f5eb3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.511899 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.564480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11536f2-c278-46cf-88e7-ef26ee5abd05 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.596270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e430c-ba42-4b79-ac5e-aeccd80f5eb3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.598020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e1bd9-5011-438a-b205-97ac6823b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.598658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e1bd9-5011-438a-b205-97ac6823b96c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.598944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e1bd9-5011-438a-b205-97ac6823b96c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.599224 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e1bd9-5011-438a-b205-97ac6823b96c None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.600363 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e1bd9-5011-438a-b205-97ac6823b96c None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.601856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e1bd9-5011-438a-b205-97ac6823b96c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.603531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 40/76] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.604293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e1bd9-5011-438a-b205-97ac6823b96c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.635852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66a2a4-57c2-451d-b942-eaa85fd5cb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.636272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66a2a4-57c2-451d-b942-eaa85fd5cb8d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.636551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66a2a4-57c2-451d-b942-eaa85fd5cb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.636822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66a2a4-57c2-451d-b942-eaa85fd5cb8d None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.638097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66a2a4-57c2-451d-b942-eaa85fd5cb8d None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.639548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66a2a4-57c2-451d-b942-eaa85fd5cb8d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.641228 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 37/77] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.641991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66a2a4-57c2-451d-b942-eaa85fd5cb8d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.682446 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.709027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e1bd9-5011-438a-b205-97ac6823b96c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.728818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.742665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c78a7-9834-43af-a858-5c20bc517ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.743302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c78a7-9834-43af-a858-5c20bc517ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.743588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c78a7-9834-43af-a858-5c20bc517ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.743871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c78a7-9834-43af-a858-5c20bc517ad7 None admin] PATH_INFO: `/v3/users/98eb25bc1113448eb680b306b130079c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.744022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66a2a4-57c2-451d-b942-eaa85fd5cb8d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.745009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c78a7-9834-43af-a858-5c20bc517ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=98eb25bc1113448eb680b306b130079c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.746518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c78a7-9834-43af-a858-5c20bc517ad7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.748373 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 41/78] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/users/98eb25bc1113448eb680b306b130079c => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.753144 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c78a7-9834-43af-a858-5c20bc517ad7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.780540 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c07d2f-a664-4ddd-b84f-a56b8784fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.781173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c07d2f-a664-4ddd-b84f-a56b8784fb33 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.781488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c07d2f-a664-4ddd-b84f-a56b8784fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.781768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c07d2f-a664-4ddd-b84f-a56b8784fb33 None admin] PATH_INFO: `/v3/projects/44a1ee379e9f470a9579bec252c86c9e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.782902 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c07d2f-a664-4ddd-b84f-a56b8784fb33 None admin] RBAC: Authorizing `identity:get_project(project_id=44a1ee379e9f470a9579bec252c86c9e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.784612 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c07d2f-a664-4ddd-b84f-a56b8784fb33 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.786452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 38/79] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/projects/44a1ee379e9f470a9579bec252c86c9e => generated 312 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.787008 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c07d2f-a664-4ddd-b84f-a56b8784fb33 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.851185 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c78a7-9834-43af-a858-5c20bc517ad7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.884270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca94c-9750-4ef7-96af-b570096bbc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.884879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca94c-9750-4ef7-96af-b570096bbc78 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.885167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca94c-9750-4ef7-96af-b570096bbc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.885484 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca94c-9750-4ef7-96af-b570096bbc78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.886585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca94c-9750-4ef7-96af-b570096bbc78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.888197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca94c-9750-4ef7-96af-b570096bbc78 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.888301 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c07d2f-a664-4ddd-b84f-a56b8784fb33 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:55.889807 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 42/80] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:55.894714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca94c-9750-4ef7-96af-b570096bbc78 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:55.895765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.923442 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac5fcd-410f-4c4c-b9bd-fbbc817df69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:55.924108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5fcd-410f-4c4c-b9bd-fbbc817df69f None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:55.924388 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5fcd-410f-4c4c-b9bd-fbbc817df69f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:55.924667 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5fcd-410f-4c4c-b9bd-fbbc817df69f None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/users/98eb25bc1113448eb680b306b130079c/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:55.926691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac5fcd-410f-4c4c-b9bd-fbbc817df69f None admin] RBAC: Authorizing `identity:create_grant(project_id=a1c030436e634b6bb7d8548e2f2e6e41, user_id=98eb25bc1113448eb680b306b130079c, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:55.928117 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac5fcd-410f-4c4c-b9bd-fbbc817df69f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:55.951251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:55.977507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 39/81] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:42:55 2022] PUT /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/users/98eb25bc1113448eb680b306b130079c/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:42:55.986759 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac5fcd-410f-4c4c-b9bd-fbbc817df69f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:56.013243 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca94c-9750-4ef7-96af-b570096bbc78 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:56.053897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c370f2-0fb3-4ba7-93ef-a6a53a557894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:56.054581 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c370f2-0fb3-4ba7-93ef-a6a53a557894 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:56.054863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c370f2-0fb3-4ba7-93ef-a6a53a557894 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:56.055134 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c370f2-0fb3-4ba7-93ef-a6a53a557894 None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/users/05701e65a5f34ab3837cd0f450bded1e/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:56.057698 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c370f2-0fb3-4ba7-93ef-a6a53a557894 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1c030436e634b6bb7d8548e2f2e6e41, user_id=05701e65a5f34ab3837cd0f450bded1e, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:56.059193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c370f2-0fb3-4ba7-93ef-a6a53a557894 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:56.091749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac5fcd-410f-4c4c-b9bd-fbbc817df69f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:56.130208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2658a7b-f512-4e81-b2a1-15360070044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:56.130800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2658a7b-f512-4e81-b2a1-15360070044f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:56.131087 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2658a7b-f512-4e81-b2a1-15360070044f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:56.131368 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2658a7b-f512-4e81-b2a1-15360070044f None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:56.141600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 43/82] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:42:55 2022] PUT /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/users/05701e65a5f34ab3837cd0f450bded1e/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:42:56.141787 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2658a7b-f512-4e81-b2a1-15360070044f None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:56.145041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c370f2-0fb3-4ba7-93ef-a6a53a557894 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:56.145212 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2658a7b-f512-4e81-b2a1-15360070044f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:56.145212 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 40/83] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:42:55 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:56.145212 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2658a7b-f512-4e81-b2a1-15360070044f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:56.196039 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:56.289953 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2658a7b-f512-4e81-b2a1-15360070044f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:56.290169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c370f2-0fb3-4ba7-93ef-a6a53a557894 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:56.313903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:56.333013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e017be7-7da1-4aa5-af5a-8035bc496589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:56.333664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e017be7-7da1-4aa5-af5a-8035bc496589 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:56.334071 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e017be7-7da1-4aa5-af5a-8035bc496589 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:56.334354 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e017be7-7da1-4aa5-af5a-8035bc496589 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:56.335017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e017be7-7da1-4aa5-af5a-8035bc496589 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:56.336531 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e017be7-7da1-4aa5-af5a-8035bc496589 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:56.339388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8e6dd-01b3-46e3-adb7-befbf5a85ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:56.340240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8e6dd-01b3-46e3-adb7-befbf5a85ec1 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:56.340676 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8e6dd-01b3-46e3-adb7-befbf5a85ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:56.341638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8e6dd-01b3-46e3-adb7-befbf5a85ec1 None admin] PATH_INFO: `/v3/projects/44a1ee379e9f470a9579bec252c86c9e/users/98eb25bc1113448eb680b306b130079c/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:56.345433 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8e6dd-01b3-46e3-adb7-befbf5a85ec1 None admin] RBAC: Authorizing `identity:create_grant(project_id=44a1ee379e9f470a9579bec252c86c9e, user_id=98eb25bc1113448eb680b306b130079c, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:56.347559 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8e6dd-01b3-46e3-adb7-befbf5a85ec1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:56.401339 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 41/84] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:42:56 2022] POST /identity/v3/users => generated 289 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:42:56.401939 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e017be7-7da1-4aa5-af5a-8035bc496589 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:56.426530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 44/85] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:42:56 2022] PUT /identity/v3/projects/44a1ee379e9f470a9579bec252c86c9e/users/98eb25bc1113448eb680b306b130079c/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:42:56.521467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:56.537593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e017be7-7da1-4aa5-af5a-8035bc496589 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:56.593833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a9496-c03b-4205-8bc9-ad7e492fcb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:56.594541 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a9496-c03b-4205-8bc9-ad7e492fcb5f None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:56.594833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a9496-c03b-4205-8bc9-ad7e492fcb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:56.595117 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a9496-c03b-4205-8bc9-ad7e492fcb5f None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/users/98eb25bc1113448eb680b306b130079c/roles/ffd57ebcb27d45ef9b9d642f409cc6d7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:56.598838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a9496-c03b-4205-8bc9-ad7e492fcb5f None admin] RBAC: Authorizing `identity:create_grant(project_id=a1c030436e634b6bb7d8548e2f2e6e41, user_id=98eb25bc1113448eb680b306b130079c, role_id=ffd57ebcb27d45ef9b9d642f409cc6d7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:56.600396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a9496-c03b-4205-8bc9-ad7e492fcb5f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:56.665589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 42/86] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:42:56 2022] PUT /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/users/98eb25bc1113448eb680b306b130079c/roles/ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:42:58.465210 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdea17-a154-463e-826f-bcb54b677fe1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:58.466302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdea17-a154-463e-826f-bcb54b677fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:58.466943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdea17-a154-463e-826f-bcb54b677fe1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:58.468515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 45/87] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:58.474214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102810f5-01e0-4e83-aeb1-9f5d78c5d80b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:58.474642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102810f5-01e0-4e83-aeb1-9f5d78c5d80b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:58.474835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102810f5-01e0-4e83-aeb1-9f5d78c5d80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:58.498248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-102810f5-01e0-4e83-aeb1-9f5d78c5d80b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:58.499495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-102810f5-01e0-4e83-aeb1-9f5d78c5d80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:58.598519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 43/88] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:58.807694 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57786a4-29cd-4e7a-9ba2-eebf75cb0d04 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:58.808385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57786a4-29cd-4e7a-9ba2-eebf75cb0d04 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:58.808866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57786a4-29cd-4e7a-9ba2-eebf75cb0d04 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:58.811216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 46/89] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:58.816662 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b61dda-aa36-4a8a-9142-340703ba189c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:58.817227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b61dda-aa36-4a8a-9142-340703ba189c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:58.817740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b61dda-aa36-4a8a-9142-340703ba189c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:58.839859 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c4b61dda-aa36-4a8a-9142-340703ba189c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:58.845984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c4b61dda-aa36-4a8a-9142-340703ba189c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:58.935252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 44/90] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:59.085665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f2851-c5e6-441d-a2ad-2ee2a50a71bf None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:59.086276 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f2851-c5e6-441d-a2ad-2ee2a50a71bf None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:59.086754 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f2851-c5e6-441d-a2ad-2ee2a50a71bf None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:59.088316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 47/91] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:42:59.093776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bb548-9621-4338-97ea-d3ead0a23e39 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:59.094296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bb548-9621-4338-97ea-d3ead0a23e39 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:59.094791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bb548-9621-4338-97ea-d3ead0a23e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:59.117795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-108bb548-9621-4338-97ea-d3ead0a23e39 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:42:59.119294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-108bb548-9621-4338-97ea-d3ead0a23e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:42:59.126132 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:59.193361 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f2851-c5e6-441d-a2ad-2ee2a50a71bf None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:59.219200 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 45/92] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:42:59.339973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f2851-c5e6-441d-a2ad-2ee2a50a71bf None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:59.347846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bb548-9621-4338-97ea-d3ead0a23e39 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:59.406605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717adf41-92ef-44c4-9da9-c0ae58c4ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:59.407262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717adf41-92ef-44c4-9da9-c0ae58c4ec73 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:59.407546 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717adf41-92ef-44c4-9da9-c0ae58c4ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:59.407825 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717adf41-92ef-44c4-9da9-c0ae58c4ec73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:59.426181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717adf41-92ef-44c4-9da9-c0ae58c4ec73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-717adf41-92ef-44c4-9da9-c0ae58c4ec73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:42:59.431994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:59.432880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:42:59.433674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 48/93] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:42:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:42:59.439774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717adf41-92ef-44c4-9da9-c0ae58c4ec73 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:59.511805 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:59.524401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bb548-9621-4338-97ea-d3ead0a23e39 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:59.568887 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46cccd-4193-4837-a881-9a99a8f4f5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:59.569565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46cccd-4193-4837-a881-9a99a8f4f5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:59.569851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46cccd-4193-4837-a881-9a99a8f4f5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:59.570126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46cccd-4193-4837-a881-9a99a8f4f5e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:59.578937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:59.582628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46cccd-4193-4837-a881-9a99a8f4f5e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5a46cccd-4193-4837-a881-9a99a8f4f5e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:42:59.587686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:59.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 46/94] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:42:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:42:59.591291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46cccd-4193-4837-a881-9a99a8f4f5e7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:59.665412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717adf41-92ef-44c4-9da9-c0ae58c4ec73 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:59.733389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0112f1f-5bab-40a6-97a4-03ea67729bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:59.733996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0112f1f-5bab-40a6-97a4-03ea67729bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:59.734279 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0112f1f-5bab-40a6-97a4-03ea67729bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:59.734560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0112f1f-5bab-40a6-97a4-03ea67729bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:59.735238 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0112f1f-5bab-40a6-97a4-03ea67729bd7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:42:59.735545 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0112f1f-5bab-40a6-97a4-03ea67729bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:42:59.736946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0112f1f-5bab-40a6-97a4-03ea67729bd7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:42:59.743519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46cccd-4193-4837-a881-9a99a8f4f5e7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:59.757926 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 49/95] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:42:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:42:59.758817 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0112f1f-5bab-40a6-97a4-03ea67729bd7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:59.810385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acdd51-58bd-4d9e-ace7-f7210c1e3141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:42:59.811029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acdd51-58bd-4d9e-ace7-f7210c1e3141 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:42:59.811312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acdd51-58bd-4d9e-ace7-f7210c1e3141 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:42:59.811587 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acdd51-58bd-4d9e-ace7-f7210c1e3141 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:42:59.835962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acdd51-58bd-4d9e-ace7-f7210c1e3141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c4acdd51-58bd-4d9e-ace7-f7210c1e3141 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:42:59.840971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:42:59.842195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 47/96] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:42:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:42:59.843532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acdd51-58bd-4d9e-ace7-f7210c1e3141 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:42:59.953574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:42:59.960455 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0112f1f-5bab-40a6-97a4-03ea67729bd7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:42:59.976036 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:00.030503 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b84732-fd65-4d34-ade3-cf1edd81985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:00.031197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b84732-fd65-4d34-ade3-cf1edd81985b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.031662 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b84732-fd65-4d34-ade3-cf1edd81985b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.031949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b84732-fd65-4d34-ade3-cf1edd81985b None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.033617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b84732-fd65-4d34-ade3-cf1edd81985b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:00.033943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b84732-fd65-4d34-ade3-cf1edd81985b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:00.035945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b84732-fd65-4d34-ade3-cf1edd81985b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:00.041469 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acdd51-58bd-4d9e-ace7-f7210c1e3141 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:00.047742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 50/97] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:42:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:00.048607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b84732-fd65-4d34-ade3-cf1edd81985b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:00.083145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e53ac6-5268-415f-81bd-73fa60b59bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:00.084311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e53ac6-5268-415f-81bd-73fa60b59bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.085122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e53ac6-5268-415f-81bd-73fa60b59bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.085800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e53ac6-5268-415f-81bd-73fa60b59bc7 None admin] PATH_INFO: `/v3/users/98eb25bc1113448eb680b306b130079c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.087257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e53ac6-5268-415f-81bd-73fa60b59bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=98eb25bc1113448eb680b306b130079c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:00.088858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e53ac6-5268-415f-81bd-73fa60b59bc7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:00.091125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 48/98] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:42:59 2022] GET /identity/v3/users/98eb25bc1113448eb680b306b130079c => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:00.097293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55687804-c6e0-44c4-b399-e72455f686e9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.097293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55687804-c6e0-44c4-b399-e72455f686e9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.097293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55687804-c6e0-44c4-b399-e72455f686e9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.097293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 49/99] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:00.097293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55687804-c6e0-44c4-b399-e72455f686e9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:00.167033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b84732-fd65-4d34-ade3-cf1edd81985b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:00.204813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25650804-4c16-43a8-b4f3-2cff9c253921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:00.206146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25650804-4c16-43a8-b4f3-2cff9c253921 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.206896 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25650804-4c16-43a8-b4f3-2cff9c253921 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.207380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25650804-4c16-43a8-b4f3-2cff9c253921 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.208696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25650804-4c16-43a8-b4f3-2cff9c253921 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:00.209499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25650804-4c16-43a8-b4f3-2cff9c253921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:00.211655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25650804-4c16-43a8-b4f3-2cff9c253921 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:00.215279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55687804-c6e0-44c4-b399-e72455f686e9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:00.222078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 51/100] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:00.223262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25650804-4c16-43a8-b4f3-2cff9c253921 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:00.252328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:00.252643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:00.261171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8f5e1-bf11-40be-b6d5-81edd42cc8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:00.261839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8f5e1-bf11-40be-b6d5-81edd42cc8db None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.262118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8f5e1-bf11-40be-b6d5-81edd42cc8db None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.269840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8f5e1-bf11-40be-b6d5-81edd42cc8db None admin] PATH_INFO: `/v3/users/05701e65a5f34ab3837cd0f450bded1e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.269840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8f5e1-bf11-40be-b6d5-81edd42cc8db None admin] RBAC: Authorizing `identity:get_user(user_id=05701e65a5f34ab3837cd0f450bded1e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:00.269840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8f5e1-bf11-40be-b6d5-81edd42cc8db None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:00.269840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 50/101] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:00 2022] GET /identity/v3/users/05701e65a5f34ab3837cd0f450bded1e => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:00.269840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412e5c2-0c09-4678-9aa2-4fdf344126a1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.269840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412e5c2-0c09-4678-9aa2-4fdf344126a1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.270229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412e5c2-0c09-4678-9aa2-4fdf344126a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.287363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9412e5c2-0c09-4678-9aa2-4fdf344126a1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:00.288560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9412e5c2-0c09-4678-9aa2-4fdf344126a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:00.334689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25650804-4c16-43a8-b4f3-2cff9c253921 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:00.365327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 51/102] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:00.365327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1b0ed-a05f-48b3-8486-0ad9a55496ce None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.365327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1b0ed-a05f-48b3-8486-0ad9a55496ce None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.365327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1b0ed-a05f-48b3-8486-0ad9a55496ce None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.365327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 52/103] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:00.365327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1b0ed-a05f-48b3-8486-0ad9a55496ce None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:00.381334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d37f2-8cc6-489e-b50d-718397d5fa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:00.382009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d37f2-8cc6-489e-b50d-718397d5fa92 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.382293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d37f2-8cc6-489e-b50d-718397d5fa92 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.382568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d37f2-8cc6-489e-b50d-718397d5fa92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.455354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d37f2-8cc6-489e-b50d-718397d5fa92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:00.456856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d37f2-8cc6-489e-b50d-718397d5fa92 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:00.480595 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:00.490438 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1b0ed-a05f-48b3-8486-0ad9a55496ce None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:00.526113 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:00.539449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7633e1e-80d3-4a41-996b-c4e4c880ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:00.540143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633e1e-80d3-4a41-996b-c4e4c880ffce None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.540838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633e1e-80d3-4a41-996b-c4e4c880ffce None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.540838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633e1e-80d3-4a41-996b-c4e4c880ffce None admin] PATH_INFO: `/v3/users/98eb25bc1113448eb680b306b130079c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.550345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7633e1e-80d3-4a41-996b-c4e4c880ffce None admin] RBAC: Authorizing `identity:get_user(user_id=98eb25bc1113448eb680b306b130079c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:00.552562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7633e1e-80d3-4a41-996b-c4e4c880ffce None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:00.554353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 53/104] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:00 2022] GET /identity/v3/users/98eb25bc1113448eb680b306b130079c => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:00.555144 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7633e1e-80d3-4a41-996b-c4e4c880ffce None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:00.575469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 52/105] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:00.577465 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86e767-642d-4cc4-a40d-d94e672f6304 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.577956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86e767-642d-4cc4-a40d-d94e672f6304 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.578431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86e767-642d-4cc4-a40d-d94e672f6304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.597124 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8b86e767-642d-4cc4-a40d-d94e672f6304 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:00.598699 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8b86e767-642d-4cc4-a40d-d94e672f6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:00.674172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 53/106] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:00.674944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86e767-642d-4cc4-a40d-d94e672f6304 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:00.680439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7633e1e-80d3-4a41-996b-c4e4c880ffce None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:00.718406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c25d80-9852-4efe-8d74-6d05a2a1668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:00.719350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c25d80-9852-4efe-8d74-6d05a2a1668f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.719636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c25d80-9852-4efe-8d74-6d05a2a1668f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.719923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c25d80-9852-4efe-8d74-6d05a2a1668f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.727993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:00.775355 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86e767-642d-4cc4-a40d-d94e672f6304 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:00.794862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:00.803885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c25d80-9852-4efe-8d74-6d05a2a1668f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:00.805413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c25d80-9852-4efe-8d74-6d05a2a1668f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:00.810941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c049a79-b650-45f1-91b1-e3e7358a301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:00.811559 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049a79-b650-45f1-91b1-e3e7358a301a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.811841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049a79-b650-45f1-91b1-e3e7358a301a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.812208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049a79-b650-45f1-91b1-e3e7358a301a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.881422 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c049a79-b650-45f1-91b1-e3e7358a301a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:00.882909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c049a79-b650-45f1-91b1-e3e7358a301a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:00.918890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 54/107] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:00.920064 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c25d80-9852-4efe-8d74-6d05a2a1668f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:00.942146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:00.991390 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 54/108] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:00.993022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a846bf-6590-4679-af1b-e0757f2a94c2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:00.993507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a846bf-6590-4679-af1b-e0757f2a94c2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:00.993793 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a846bf-6590-4679-af1b-e0757f2a94c2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:00.995192 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 55/109] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:00.996001 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a846bf-6590-4679-af1b-e0757f2a94c2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.019485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:01.026041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c25d80-9852-4efe-8d74-6d05a2a1668f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.062069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45fd87-70fc-4eef-8658-1d116ac7bfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.062702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45fd87-70fc-4eef-8658-1d116ac7bfaa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.062986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45fd87-70fc-4eef-8658-1d116ac7bfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.063264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45fd87-70fc-4eef-8658-1d116ac7bfaa None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.064382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45fd87-70fc-4eef-8658-1d116ac7bfaa None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:01.065872 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45fd87-70fc-4eef-8658-1d116ac7bfaa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:01.067503 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 55/110] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:00 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:01.068273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45fd87-70fc-4eef-8658-1d116ac7bfaa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.103359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a846bf-6590-4679-af1b-e0757f2a94c2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.138710 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914d260-f813-4100-8b23-9d0b47f47d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.139350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d260-f813-4100-8b23-9d0b47f47d59 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.139639 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d260-f813-4100-8b23-9d0b47f47d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.139918 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d260-f813-4100-8b23-9d0b47f47d59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.150322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914d260-f813-4100-8b23-9d0b47f47d59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f914d260-f813-4100-8b23-9d0b47f47d59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:01.155767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:01.156693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:01.157594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 56/111] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:01.158228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914d260-f813-4100-8b23-9d0b47f47d59 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.168412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45fd87-70fc-4eef-8658-1d116ac7bfaa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.173893 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:01.203036 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0676bdd-5f17-4c10-9c23-f1c0442bbefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.203604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0676bdd-5f17-4c10-9c23-f1c0442bbefa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.203887 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0676bdd-5f17-4c10-9c23-f1c0442bbefa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.204166 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0676bdd-5f17-4c10-9c23-f1c0442bbefa None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.205312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0676bdd-5f17-4c10-9c23-f1c0442bbefa None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:01.206786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0676bdd-5f17-4c10-9c23-f1c0442bbefa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:01.208381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 56/112] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:01.209146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0676bdd-5f17-4c10-9c23-f1c0442bbefa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.240361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:01.260379 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914d260-f813-4100-8b23-9d0b47f47d59 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.293483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83715fc0-6037-4ac8-aace-d55739cce1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.295150 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83715fc0-6037-4ac8-aace-d55739cce1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.295444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83715fc0-6037-4ac8-aace-d55739cce1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.295727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83715fc0-6037-4ac8-aace-d55739cce1d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.305888 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83715fc0-6037-4ac8-aace-d55739cce1d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:01.310365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0676bdd-5f17-4c10-9c23-f1c0442bbefa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-83715fc0-6037-4ac8-aace-d55739cce1d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:01.311003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:01.311940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:01.312702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 57/113] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:01.313523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345f64e-066a-49c9-9dc1-8fb6b9ffd14f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.313588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345f64e-066a-49c9-9dc1-8fb6b9ffd14f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.313999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345f64e-066a-49c9-9dc1-8fb6b9ffd14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.331461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3345f64e-066a-49c9-9dc1-8fb6b9ffd14f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:01.332686 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3345f64e-066a-49c9-9dc1-8fb6b9ffd14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:01.348875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f30d4-1767-4fe7-a359-2430641a80df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.349528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f30d4-1767-4fe7-a359-2430641a80df None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.349807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f30d4-1767-4fe7-a359-2430641a80df None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.350088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f30d4-1767-4fe7-a359-2430641a80df None admin] PATH_INFO: `/v3/projects/44a1ee379e9f470a9579bec252c86c9e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.351198 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f30d4-1767-4fe7-a359-2430641a80df None admin] RBAC: Authorizing `identity:get_project(project_id=44a1ee379e9f470a9579bec252c86c9e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:01.352624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f30d4-1767-4fe7-a359-2430641a80df None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:01.354284 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 57/114] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/projects/44a1ee379e9f470a9579bec252c86c9e => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:01.355118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f30d4-1767-4fe7-a359-2430641a80df None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.399487 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 58/115] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:01.400417 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345f64e-066a-49c9-9dc1-8fb6b9ffd14f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.403450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:01.455518 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:01.460846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f30d4-1767-4fe7-a359-2430641a80df None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.499401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaf507-5268-4385-b7f9-6ff2d7a2b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.500833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaf507-5268-4385-b7f9-6ff2d7a2b1ab None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.501352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaf507-5268-4385-b7f9-6ff2d7a2b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.501798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaf507-5268-4385-b7f9-6ff2d7a2b1ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.502539 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345f64e-066a-49c9-9dc1-8fb6b9ffd14f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.503200 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaf507-5268-4385-b7f9-6ff2d7a2b1ab None admin] RBAC: Adding query filter params (role.id=f1fda4fad003483dbc5843e16b6dc9d2, scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41, user.id=98eb25bc1113448eb680b306b130079c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:01.504336 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaf507-5268-4385-b7f9-6ff2d7a2b1ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:01.506731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaf507-5268-4385-b7f9-6ff2d7a2b1ab None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:01.514236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 58/116] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/role_assignments?user.id=98eb25bc1113448eb680b306b130079c&scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41&role.id=f1fda4fad003483dbc5843e16b6dc9d2 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:01.515497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaf507-5268-4385-b7f9-6ff2d7a2b1ab None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.542420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155abf6-6145-42cc-a239-dc2fe4bcd7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.543068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155abf6-6145-42cc-a239-dc2fe4bcd7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.543357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155abf6-6145-42cc-a239-dc2fe4bcd7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.543640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155abf6-6145-42cc-a239-dc2fe4bcd7d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.545139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155abf6-6145-42cc-a239-dc2fe4bcd7d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:01.545488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155abf6-6145-42cc-a239-dc2fe4bcd7d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:01.546938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155abf6-6145-42cc-a239-dc2fe4bcd7d4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:01.564687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 59/117] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:01.564687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155abf6-6145-42cc-a239-dc2fe4bcd7d4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.651299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:01.672246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaf507-5268-4385-b7f9-6ff2d7a2b1ab None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.703207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155abf6-6145-42cc-a239-dc2fe4bcd7d4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.714235 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c16e5-ed60-42ce-a7ce-22227a607f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.714783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c16e5-ed60-42ce-a7ce-22227a607f3d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.715070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c16e5-ed60-42ce-a7ce-22227a607f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.715354 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c16e5-ed60-42ce-a7ce-22227a607f3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.716124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c16e5-ed60-42ce-a7ce-22227a607f3d None admin] RBAC: Adding query filter params (role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41, user.id=05701e65a5f34ab3837cd0f450bded1e) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:01.716436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c16e5-ed60-42ce-a7ce-22227a607f3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:01.717986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c16e5-ed60-42ce-a7ce-22227a607f3d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:01.725051 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 59/118] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/role_assignments?user.id=05701e65a5f34ab3837cd0f450bded1e&scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:01.725722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c16e5-ed60-42ce-a7ce-22227a607f3d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.749313 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc9683-139f-41c4-aa81-4110a8808526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.749954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc9683-139f-41c4-aa81-4110a8808526 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.750246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc9683-139f-41c4-aa81-4110a8808526 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.750530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc9683-139f-41c4-aa81-4110a8808526 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.751239 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc9683-139f-41c4-aa81-4110a8808526 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:01.751555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc9683-139f-41c4-aa81-4110a8808526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:01.753016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc9683-139f-41c4-aa81-4110a8808526 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:01.763995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 60/119] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:01.764867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc9683-139f-41c4-aa81-4110a8808526 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.768799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:01.850742 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c16e5-ed60-42ce-a7ce-22227a607f3d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.879683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc9683-139f-41c4-aa81-4110a8808526 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:01.888703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9bf77-8927-4f69-9f80-a5858747d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.889369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9bf77-8927-4f69-9f80-a5858747d73e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.889662 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9bf77-8927-4f69-9f80-a5858747d73e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.890005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9bf77-8927-4f69-9f80-a5858747d73e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.890775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9bf77-8927-4f69-9f80-a5858747d73e None admin] RBAC: Adding query filter params (role.id=f1fda4fad003483dbc5843e16b6dc9d2, scope.project.id=44a1ee379e9f470a9579bec252c86c9e, user.id=98eb25bc1113448eb680b306b130079c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:01.891091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9bf77-8927-4f69-9f80-a5858747d73e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:01.892594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9bf77-8927-4f69-9f80-a5858747d73e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:01.899131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 60/120] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/role_assignments?user.id=98eb25bc1113448eb680b306b130079c&scope.project.id=44a1ee379e9f470a9579bec252c86c9e&role.id=f1fda4fad003483dbc5843e16b6dc9d2 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:01.899793 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9bf77-8927-4f69-9f80-a5858747d73e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:01.910907 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:01.921391 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43ead4-37a8-4640-8097-967a69471232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:01.922065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43ead4-37a8-4640-8097-967a69471232 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:01.922353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43ead4-37a8-4640-8097-967a69471232 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:01.922636 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43ead4-37a8-4640-8097-967a69471232 None admin] PATH_INFO: `/v3/users/9db36e4492fc4bd3b68cec1457056734` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:01.939097 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43ead4-37a8-4640-8097-967a69471232 None admin] RBAC: Authorizing `identity:get_user(user_id=9db36e4492fc4bd3b68cec1457056734)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:01.940629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43ead4-37a8-4640-8097-967a69471232 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:01.942856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 61/121] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/users/9db36e4492fc4bd3b68cec1457056734 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:01.943719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43ead4-37a8-4640-8097-967a69471232 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:02.038503 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:02.049210 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9bf77-8927-4f69-9f80-a5858747d73e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:02.070311 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43ead4-37a8-4640-8097-967a69471232 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:02.089138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d65ea-a27c-46c1-a315-ffe907f96b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:02.089804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d65ea-a27c-46c1-a315-ffe907f96b33 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:02.090096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d65ea-a27c-46c1-a315-ffe907f96b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:02.090680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d65ea-a27c-46c1-a315-ffe907f96b33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:02.092416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d65ea-a27c-46c1-a315-ffe907f96b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:02.094035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d65ea-a27c-46c1-a315-ffe907f96b33 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:02.095799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 61/122] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:02.096236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d65ea-a27c-46c1-a315-ffe907f96b33 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:02.117086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5b75d-396d-4807-8a9d-02bfb696c769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:02.118063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5b75d-396d-4807-8a9d-02bfb696c769 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:02.118169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5b75d-396d-4807-8a9d-02bfb696c769 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:02.118395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5b75d-396d-4807-8a9d-02bfb696c769 None admin] PATH_INFO: `/v3/users/98eb25bc1113448eb680b306b130079c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:02.119539 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5b75d-396d-4807-8a9d-02bfb696c769 None admin] RBAC: Authorizing `identity:get_user(user_id=98eb25bc1113448eb680b306b130079c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:02.120997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5b75d-396d-4807-8a9d-02bfb696c769 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:02.123472 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 62/123] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:01 2022] GET /identity/v3/users/98eb25bc1113448eb680b306b130079c => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:02.124154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5b75d-396d-4807-8a9d-02bfb696c769 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:02.179158 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:02.202936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d65ea-a27c-46c1-a315-ffe907f96b33 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:02.225809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5b75d-396d-4807-8a9d-02bfb696c769 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:02.236737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb02308-03e1-4d0d-b5ca-194131c8e643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:02.237418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02308-03e1-4d0d-b5ca-194131c8e643 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:02.237703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02308-03e1-4d0d-b5ca-194131c8e643 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:02.237983 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02308-03e1-4d0d-b5ca-194131c8e643 None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:02.239109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb02308-03e1-4d0d-b5ca-194131c8e643 None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:02.240567 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb02308-03e1-4d0d-b5ca-194131c8e643 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:02.242280 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 62/124] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:02 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:02.243052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb02308-03e1-4d0d-b5ca-194131c8e643 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:02.264008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66f092-f2cc-46f8-b378-a14b3cd88412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:02.265054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66f092-f2cc-46f8-b378-a14b3cd88412 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:02.265633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66f092-f2cc-46f8-b378-a14b3cd88412 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:02.266145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66f092-f2cc-46f8-b378-a14b3cd88412 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:02.267064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66f092-f2cc-46f8-b378-a14b3cd88412 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:02.269269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66f092-f2cc-46f8-b378-a14b3cd88412 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:02.274380 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:02.346974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb02308-03e1-4d0d-b5ca-194131c8e643 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:02.363228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 63/125] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:43:02 2022] POST /identity/v3/projects => generated 302 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:02.364139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66f092-f2cc-46f8-b378-a14b3cd88412 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:02.381968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d675757-b30f-4d29-9df0-b1592b235616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:02.382613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d675757-b30f-4d29-9df0-b1592b235616 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:02.382931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d675757-b30f-4d29-9df0-b1592b235616 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:02.391139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d675757-b30f-4d29-9df0-b1592b235616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:02.467715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d675757-b30f-4d29-9df0-b1592b235616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:02.469956 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d675757-b30f-4d29-9df0-b1592b235616 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:02.504814 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:02.507961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66f092-f2cc-46f8-b378-a14b3cd88412 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:02.510932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:02.551951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba22c9c-d8eb-4de1-be61-8a6f6450bd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:02.553220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22c9c-d8eb-4de1-be61-8a6f6450bd53 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:02.553220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22c9c-d8eb-4de1-be61-8a6f6450bd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:02.553220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22c9c-d8eb-4de1-be61-8a6f6450bd53 None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/users/9db36e4492fc4bd3b68cec1457056734/roles/8b0de894383f4c18a3363436c36841fa` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:02.556509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba22c9c-d8eb-4de1-be61-8a6f6450bd53 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1c030436e634b6bb7d8548e2f2e6e41, user_id=9db36e4492fc4bd3b68cec1457056734, role_id=8b0de894383f4c18a3363436c36841fa)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:02.559048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba22c9c-d8eb-4de1-be61-8a6f6450bd53 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:02.592241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 63/126] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:02.598569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d675757-b30f-4d29-9df0-b1592b235616 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:02.601759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 64/127] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:02 2022] PUT /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/users/9db36e4492fc4bd3b68cec1457056734/roles/8b0de894383f4c18a3363436c36841fa => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:02.718845 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d675757-b30f-4d29-9df0-b1592b235616 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:02.750161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:02.759983 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6711ab6-4629-4f3d-89cc-4ddd1715865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:02.760813 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6711ab6-4629-4f3d-89cc-4ddd1715865b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:02.761116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6711ab6-4629-4f3d-89cc-4ddd1715865b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:02.761433 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6711ab6-4629-4f3d-89cc-4ddd1715865b None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:02.762675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6711ab6-4629-4f3d-89cc-4ddd1715865b None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:02.764919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6711ab6-4629-4f3d-89cc-4ddd1715865b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:02.766864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 64/128] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:02 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:02.773487 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba22c9c-d8eb-4de1-be61-8a6f6450bd53 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:02.881226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba22c9c-d8eb-4de1-be61-8a6f6450bd53 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:02.918333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d8aac-3cc1-4aae-83e5-d0b18df89b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:02.919660 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d8aac-3cc1-4aae-83e5-d0b18df89b6d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:02.919961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d8aac-3cc1-4aae-83e5-d0b18df89b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:02.920285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d8aac-3cc1-4aae-83e5-d0b18df89b6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:02.921034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d8aac-3cc1-4aae-83e5-d0b18df89b6d None admin] RBAC: Adding query filter params (role.id=ffd57ebcb27d45ef9b9d642f409cc6d7, scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41, user.id=98eb25bc1113448eb680b306b130079c) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:02.921375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d8aac-3cc1-4aae-83e5-d0b18df89b6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:02.922856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d8aac-3cc1-4aae-83e5-d0b18df89b6d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:02.930413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 65/129] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:02 2022] GET /identity/v3/role_assignments?user.id=98eb25bc1113448eb680b306b130079c&scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41&role.id=ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:04.552702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4d142-3872-4477-93c5-ac34270693c9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:04.553286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4d142-3872-4477-93c5-ac34270693c9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:04.553286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4d142-3872-4477-93c5-ac34270693c9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:04.555109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 65/130] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:04.560712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfd596-5f1e-4844-8954-c20db398fb3e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:04.561126 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfd596-5f1e-4844-8954-c20db398fb3e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:04.561360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfd596-5f1e-4844-8954-c20db398fb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:04.579508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d2bfd596-5f1e-4844-8954-c20db398fb3e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:04.580709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d2bfd596-5f1e-4844-8954-c20db398fb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:04.603028 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:04.648062 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 66/131] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:04.752679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d78d9b-053b-4282-904b-3b3a625a8a82 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:04.752964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d78d9b-053b-4282-904b-3b3a625a8a82 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:04.753272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d78d9b-053b-4282-904b-3b3a625a8a82 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:04.754626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 66/132] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:04.759841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723930b-992e-4577-9a40-07e9a813eb6a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:04.760135 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723930b-992e-4577-9a40-07e9a813eb6a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:04.760415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723930b-992e-4577-9a40-07e9a813eb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:04.778877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4723930b-992e-4577-9a40-07e9a813eb6a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:04.780048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4723930b-992e-4577-9a40-07e9a813eb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:04.844033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 67/133] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:04.917369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d78d9b-053b-4282-904b-3b3a625a8a82 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:05.015627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d78d9b-053b-4282-904b-3b3a625a8a82 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:05.048810 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f895e6c-5534-44ec-b621-f6726bad7df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:05.049494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f895e6c-5534-44ec-b621-f6726bad7df7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:05.049781 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f895e6c-5534-44ec-b621-f6726bad7df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:05.050061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f895e6c-5534-44ec-b621-f6726bad7df7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:05.051160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f895e6c-5534-44ec-b621-f6726bad7df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:05.052600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f895e6c-5534-44ec-b621-f6726bad7df7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:05.054389 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 67/134] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:05.059706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723930b-992e-4577-9a40-07e9a813eb6a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:05.112698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f895e6c-5534-44ec-b621-f6726bad7df7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:05.145429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:05.157477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723930b-992e-4577-9a40-07e9a813eb6a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:05.175696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:05.190672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06da02-b946-4e3b-8d10-1590b727eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:05.193823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06da02-b946-4e3b-8d10-1590b727eeea None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:05.193823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06da02-b946-4e3b-8d10-1590b727eeea None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:05.193823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06da02-b946-4e3b-8d10-1590b727eeea None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:05.193823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06da02-b946-4e3b-8d10-1590b727eeea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:05.194224 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06da02-b946-4e3b-8d10-1590b727eeea None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:05.212697 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f895e6c-5534-44ec-b621-f6726bad7df7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:05.247960 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee48b83-905b-4b2e-ae54-8dade892a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:05.248597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee48b83-905b-4b2e-ae54-8dade892a9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:05.248883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee48b83-905b-4b2e-ae54-8dade892a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:05.249165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee48b83-905b-4b2e-ae54-8dade892a9a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:05.259044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee48b83-905b-4b2e-ae54-8dade892a9a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3ee48b83-905b-4b2e-ae54-8dade892a9a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:05.264786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:05.265862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:05.266672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 68/135] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:05.271787 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee48b83-905b-4b2e-ae54-8dade892a9a6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:05.301218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 68/136] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:05 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:05.380664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee48b83-905b-4b2e-ae54-8dade892a9a6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:05.405596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:05.417886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab103c3c-4867-4d19-9e78-1b3b76d22a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:05.425441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab103c3c-4867-4d19-9e78-1b3b76d22a74 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:05.425441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab103c3c-4867-4d19-9e78-1b3b76d22a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:05.425441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab103c3c-4867-4d19-9e78-1b3b76d22a74 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:05.425441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab103c3c-4867-4d19-9e78-1b3b76d22a74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:05.425441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab103c3c-4867-4d19-9e78-1b3b76d22a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:05.425441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab103c3c-4867-4d19-9e78-1b3b76d22a74 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:05.431481 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 69/137] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:05.438423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06da02-b946-4e3b-8d10-1590b727eeea None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:05.542244 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06da02-b946-4e3b-8d10-1590b727eeea None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:05.578439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f64e8f-cc50-424d-aec3-e617060d5b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:05.579090 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f64e8f-cc50-424d-aec3-e617060d5b75 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:05.579372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f64e8f-cc50-424d-aec3-e617060d5b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:05.579646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f64e8f-cc50-424d-aec3-e617060d5b75 None admin] PATH_INFO: `/v3/users/9db36e4492fc4bd3b68cec1457056734` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:05.580816 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f64e8f-cc50-424d-aec3-e617060d5b75 None admin] RBAC: Authorizing `identity:get_user(user_id=9db36e4492fc4bd3b68cec1457056734)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:05.582383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f64e8f-cc50-424d-aec3-e617060d5b75 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:05.584201 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 69/138] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:05 2022] GET /identity/v3/users/9db36e4492fc4bd3b68cec1457056734 => generated 289 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:05.589978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab103c3c-4867-4d19-9e78-1b3b76d22a74 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:05.701559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab103c3c-4867-4d19-9e78-1b3b76d22a74 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:05.744401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f608cf-4feb-4b49-b231-068213c2be85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:05.745028 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f608cf-4feb-4b49-b231-068213c2be85 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:05.745364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f608cf-4feb-4b49-b231-068213c2be85 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:05.745640 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f608cf-4feb-4b49-b231-068213c2be85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:05.803750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:05.820803 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f608cf-4feb-4b49-b231-068213c2be85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:05.822458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f608cf-4feb-4b49-b231-068213c2be85 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:05.931836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 70/139] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:05.937785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f64e8f-cc50-424d-aec3-e617060d5b75 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:05.954352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:06.047437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f64e8f-cc50-424d-aec3-e617060d5b75 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:06.083188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a822f0c-f83b-4ac6-a7e6-d32b66f165c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:06.083834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a822f0c-f83b-4ac6-a7e6-d32b66f165c3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:06.084182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a822f0c-f83b-4ac6-a7e6-d32b66f165c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:06.084463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a822f0c-f83b-4ac6-a7e6-d32b66f165c3 None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:06.085687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a822f0c-f83b-4ac6-a7e6-d32b66f165c3 None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:06.087255 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a822f0c-f83b-4ac6-a7e6-d32b66f165c3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:06.089236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 70/140] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:05 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:06.095383 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f608cf-4feb-4b49-b231-068213c2be85 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:06.187044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:06.203067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f608cf-4feb-4b49-b231-068213c2be85 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:06.239704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2570a30-bd3b-4f70-92a7-65a6cac8e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:06.240248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2570a30-bd3b-4f70-92a7-65a6cac8e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:06.240549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2570a30-bd3b-4f70-92a7-65a6cac8e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:06.240831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2570a30-bd3b-4f70-92a7-65a6cac8e0f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:06.241611 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2570a30-bd3b-4f70-92a7-65a6cac8e0f0 None admin] RBAC: Adding query filter params (role.id=8b0de894383f4c18a3363436c36841fa, scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41, user.id=9db36e4492fc4bd3b68cec1457056734) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:06.241925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2570a30-bd3b-4f70-92a7-65a6cac8e0f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:06.243464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2570a30-bd3b-4f70-92a7-65a6cac8e0f0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:06.250654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 71/141] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:06 2022] GET /identity/v3/role_assignments?user.id=9db36e4492fc4bd3b68cec1457056734&scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41&role.id=8b0de894383f4c18a3363436c36841fa => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:07.598000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612def1-c000-4196-aec7-9ae7fa3a7cc4 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:07.598576 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612def1-c000-4196-aec7-9ae7fa3a7cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:07.598576 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612def1-c000-4196-aec7-9ae7fa3a7cc4 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:07.600166 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 71/142] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:07.605795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de11b9-801b-4909-89bb-3a38d8ebf636 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:07.606510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de11b9-801b-4909-89bb-3a38d8ebf636 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:07.606510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de11b9-801b-4909-89bb-3a38d8ebf636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:07.625955 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-73de11b9-801b-4909-89bb-3a38d8ebf636 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:07.627145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-73de11b9-801b-4909-89bb-3a38d8ebf636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:07.693250 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 72/143] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:07.708182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a613b-ccd7-4803-8ed0-9471dca63f83 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:07.708477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a613b-ccd7-4803-8ed0-9471dca63f83 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:07.708739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a613b-ccd7-4803-8ed0-9471dca63f83 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:07.710327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 72/144] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:07.716383 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e7dcc-08bc-4185-8f8d-c7653c9722f7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:07.716683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e7dcc-08bc-4185-8f8d-c7653c9722f7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:07.716942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e7dcc-08bc-4185-8f8d-c7653c9722f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:07.736303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-866e7dcc-08bc-4185-8f8d-c7653c9722f7 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:07.737546 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-866e7dcc-08bc-4185-8f8d-c7653c9722f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:07.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 73/145] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:07.836961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165326e-19b9-484e-82e3-2e9925ac4c94 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:07.837129 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165326e-19b9-484e-82e3-2e9925ac4c94 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:07.837421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165326e-19b9-484e-82e3-2e9925ac4c94 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:07.838896 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 73/146] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:07.844480 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245ad8e-b75c-460c-8f47-95af3abcbff0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:07.844777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245ad8e-b75c-460c-8f47-95af3abcbff0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:07.845060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245ad8e-b75c-460c-8f47-95af3abcbff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:07.863582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d245ad8e-b75c-460c-8f47-95af3abcbff0 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:07.864760 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d245ad8e-b75c-460c-8f47-95af3abcbff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:07.866727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:07.932343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 74/147] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:07.966845 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165326e-19b9-484e-82e3-2e9925ac4c94 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.049396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:08.068573 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165326e-19b9-484e-82e3-2e9925ac4c94 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.085538 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245ad8e-b75c-460c-8f47-95af3abcbff0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.104382 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0dd5d0-3607-4612-bf0c-244d1c0421bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.104955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dd5d0-3607-4612-bf0c-244d1c0421bf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.105326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dd5d0-3607-4612-bf0c-244d1c0421bf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.105571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dd5d0-3607-4612-bf0c-244d1c0421bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.116215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0dd5d0-3607-4612-bf0c-244d1c0421bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ce0dd5d0-3607-4612-bf0c-244d1c0421bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:08.121289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:08.123022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:08.123843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 74/148] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:43:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:08.129306 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0dd5d0-3607-4612-bf0c-244d1c0421bf None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.191245 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245ad8e-b75c-460c-8f47-95af3abcbff0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.224745 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae127f2d-0d05-4c1e-8d91-ea98d5725cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.225406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae127f2d-0d05-4c1e-8d91-ea98d5725cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.225688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae127f2d-0d05-4c1e-8d91-ea98d5725cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.225958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae127f2d-0d05-4c1e-8d91-ea98d5725cf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.226698 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0dd5d0-3607-4612-bf0c-244d1c0421bf None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.232990 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:08.237449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae127f2d-0d05-4c1e-8d91-ea98d5725cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ae127f2d-0d05-4c1e-8d91-ea98d5725cf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:08.242273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:08.250756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 75/149] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:43:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:08.251709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae127f2d-0d05-4c1e-8d91-ea98d5725cf0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.260947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd19466-51df-4c63-bab7-b7c82c6e8b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.261527 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd19466-51df-4c63-bab7-b7c82c6e8b96 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.261865 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd19466-51df-4c63-bab7-b7c82c6e8b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.262139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd19466-51df-4c63-bab7-b7c82c6e8b96 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.263381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd19466-51df-4c63-bab7-b7c82c6e8b96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:08.263637 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd19466-51df-4c63-bab7-b7c82c6e8b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:08.265045 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd19466-51df-4c63-bab7-b7c82c6e8b96 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:08.273369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 75/150] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:08.278930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd19466-51df-4c63-bab7-b7c82c6e8b96 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.280423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:08.352147 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae127f2d-0d05-4c1e-8d91-ea98d5725cf0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.379183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd19466-51df-4c63-bab7-b7c82c6e8b96 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.387277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19de1-c04f-4fff-8136-f3ff145bcdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.388042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19de1-c04f-4fff-8136-f3ff145bcdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.388319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19de1-c04f-4fff-8136-f3ff145bcdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.388595 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19de1-c04f-4fff-8136-f3ff145bcdc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.389734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a19de1-c04f-4fff-8136-f3ff145bcdc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:08.391173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a19de1-c04f-4fff-8136-f3ff145bcdc0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:08.392730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 76/151] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:08.398041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19de1-c04f-4fff-8136-f3ff145bcdc0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.412516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8533e-39be-4991-8c19-f207a3c01411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.413213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8533e-39be-4991-8c19-f207a3c01411 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.413520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8533e-39be-4991-8c19-f207a3c01411 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.413795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8533e-39be-4991-8c19-f207a3c01411 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.414450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8533e-39be-4991-8c19-f207a3c01411 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:08.414755 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8533e-39be-4991-8c19-f207a3c01411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:08.416138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8533e-39be-4991-8c19-f207a3c01411 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:08.424840 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 76/152] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:08.425461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8533e-39be-4991-8c19-f207a3c01411 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.469248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:08.485052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:08.502720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19de1-c04f-4fff-8136-f3ff145bcdc0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.520591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8533e-39be-4991-8c19-f207a3c01411 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.540573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72479992-ae62-40f4-9c80-52a4d6d70aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.541274 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72479992-ae62-40f4-9c80-52a4d6d70aef None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.541553 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72479992-ae62-40f4-9c80-52a4d6d70aef None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.541827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72479992-ae62-40f4-9c80-52a4d6d70aef None admin] PATH_INFO: `/v3/users/1a4bc4998efe45f688920bef3bc6a222` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.557157 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e39c8-eba6-4486-81e8-0c130bad2b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.557399 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72479992-ae62-40f4-9c80-52a4d6d70aef None admin] RBAC: Authorizing `identity:get_user(user_id=1a4bc4998efe45f688920bef3bc6a222)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:08.557797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e39c8-eba6-4486-81e8-0c130bad2b2d None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.558079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e39c8-eba6-4486-81e8-0c130bad2b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.558354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e39c8-eba6-4486-81e8-0c130bad2b2d None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.558842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72479992-ae62-40f4-9c80-52a4d6d70aef None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:08.559013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e39c8-eba6-4486-81e8-0c130bad2b2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:08.561118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e39c8-eba6-4486-81e8-0c130bad2b2d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:08.561643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 77/153] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:08 2022] GET /identity/v3/users/1a4bc4998efe45f688920bef3bc6a222 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:08.562431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72479992-ae62-40f4-9c80-52a4d6d70aef None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.660070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72479992-ae62-40f4-9c80-52a4d6d70aef None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.682229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:08.686754 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 77/154] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:08 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:08.690124 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e39c8-eba6-4486-81e8-0c130bad2b2d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.694773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967567f4-2378-4dcb-bbd7-f99df6493398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.694773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967567f4-2378-4dcb-bbd7-f99df6493398 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.694773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967567f4-2378-4dcb-bbd7-f99df6493398 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.695065 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967567f4-2378-4dcb-bbd7-f99df6493398 None admin] PATH_INFO: `/v3/users/1a4bc4998efe45f688920bef3bc6a222` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.696260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967567f4-2378-4dcb-bbd7-f99df6493398 None admin] RBAC: Authorizing `identity:get_user(user_id=1a4bc4998efe45f688920bef3bc6a222)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:08.697792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967567f4-2378-4dcb-bbd7-f99df6493398 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:08.699478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 78/155] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:08 2022] GET /identity/v3/users/1a4bc4998efe45f688920bef3bc6a222 => generated 283 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:08.707243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967567f4-2378-4dcb-bbd7-f99df6493398 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.803260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e39c8-eba6-4486-81e8-0c130bad2b2d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.821816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967567f4-2378-4dcb-bbd7-f99df6493398 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.832599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:08.850053 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e3630-cdb0-4ce6-aa8b-fb1ad5d213a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.851273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e3630-cdb0-4ce6-aa8b-fb1ad5d213a7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.851677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e3630-cdb0-4ce6-aa8b-fb1ad5d213a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.851966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e3630-cdb0-4ce6-aa8b-fb1ad5d213a7 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.853837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e3630-cdb0-4ce6-aa8b-fb1ad5d213a7 None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:08.855921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e3630-cdb0-4ce6-aa8b-fb1ad5d213a7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:08.858057 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 78/156] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:08 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:08.859506 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa3494-eb3c-4c2a-9fb7-1530741a9368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:08.860148 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa3494-eb3c-4c2a-9fb7-1530741a9368 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:08.860425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa3494-eb3c-4c2a-9fb7-1530741a9368 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:08.860698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa3494-eb3c-4c2a-9fb7-1530741a9368 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:08.861916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa3494-eb3c-4c2a-9fb7-1530741a9368 None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:08.863943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa3494-eb3c-4c2a-9fb7-1530741a9368 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:08.864183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e3630-cdb0-4ce6-aa8b-fb1ad5d213a7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.869226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 79/157] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:08 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:08.873084 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa3494-eb3c-4c2a-9fb7-1530741a9368 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:08.942892 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:08.982383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e3630-cdb0-4ce6-aa8b-fb1ad5d213a7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:08.998713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa3494-eb3c-4c2a-9fb7-1530741a9368 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:09.022941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d747711-b145-4faa-a245-865bffd470a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:09.023713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747711-b145-4faa-a245-865bffd470a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:09.023992 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747711-b145-4faa-a245-865bffd470a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:09.024275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747711-b145-4faa-a245-865bffd470a6 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147/users/1a4bc4998efe45f688920bef3bc6a222/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:09.026986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d747711-b145-4faa-a245-865bffd470a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5580ff9cebc43c6933eb6147b391147, user_id=1a4bc4998efe45f688920bef3bc6a222, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:09.028822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d747711-b145-4faa-a245-865bffd470a6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:09.038243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a67e79-c350-4003-85c8-f60181d9b2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:09.038969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a67e79-c350-4003-85c8-f60181d9b2c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:09.039252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a67e79-c350-4003-85c8-f60181d9b2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:09.039825 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a67e79-c350-4003-85c8-f60181d9b2c0 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147/users/1a4bc4998efe45f688920bef3bc6a222/roles/ffd57ebcb27d45ef9b9d642f409cc6d7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:09.042984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a67e79-c350-4003-85c8-f60181d9b2c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5580ff9cebc43c6933eb6147b391147, user_id=1a4bc4998efe45f688920bef3bc6a222, role_id=ffd57ebcb27d45ef9b9d642f409cc6d7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:09.044827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a67e79-c350-4003-85c8-f60181d9b2c0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:09.098548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 79/158] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:08 2022] PUT /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147/users/1a4bc4998efe45f688920bef3bc6a222/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:09.128170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 80/159] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:08 2022] PUT /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147/users/1a4bc4998efe45f688920bef3bc6a222/roles/ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 12:43:10.991805 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436a952-3037-4ef9-a119-476c253c5cdd None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:10.992314 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436a952-3037-4ef9-a119-476c253c5cdd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:10.992314 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436a952-3037-4ef9-a119-476c253c5cdd None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:10.993934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 80/160] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:10.999476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315edc88-1aa9-4621-a09a-4c97e6d86787 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:10.999892 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315edc88-1aa9-4621-a09a-4c97e6d86787 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.000060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315edc88-1aa9-4621-a09a-4c97e6d86787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.018875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-315edc88-1aa9-4621-a09a-4c97e6d86787 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:11.020043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-315edc88-1aa9-4621-a09a-4c97e6d86787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:11.061107 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b65af7-78e9-47eb-a910-49445f3d801e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.061579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b65af7-78e9-47eb-a910-49445f3d801e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.061966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b65af7-78e9-47eb-a910-49445f3d801e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.063446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 81/161] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:43:11.068532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33e2b-82d1-4c32-82c2-b5560e2f9501 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.068899 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33e2b-82d1-4c32-82c2-b5560e2f9501 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.069322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33e2b-82d1-4c32-82c2-b5560e2f9501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.088631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8db33e2b-82d1-4c32-82c2-b5560e2f9501 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:11.089875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8db33e2b-82d1-4c32-82c2-b5560e2f9501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:11.091874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 81/162] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:11.136599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:11.160624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 82/163] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:11.331215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af85ec4-fa52-4978-84e0-722cc6297ab0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.331608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af85ec4-fa52-4978-84e0-722cc6297ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.331779 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af85ec4-fa52-4978-84e0-722cc6297ab0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.333427 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 82/164] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:11.338584 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15431cdb-019b-430c-af64-29848312fa7c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.339130 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15431cdb-019b-430c-af64-29848312fa7c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.339605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15431cdb-019b-430c-af64-29848312fa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.346432 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b5494-9ae0-4f6e-a3ab-1c3330206a01 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.346720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b5494-9ae0-4f6e-a3ab-1c3330206a01 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.346998 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b5494-9ae0-4f6e-a3ab-1c3330206a01 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.348430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 83/165] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:43:11.353617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7823d-5f72-4fbc-bc24-053414efbffc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.353898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7823d-5f72-4fbc-bc24-053414efbffc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.356348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7823d-5f72-4fbc-bc24-053414efbffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.364636 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-15431cdb-019b-430c-af64-29848312fa7c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:11.366479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-15431cdb-019b-430c-af64-29848312fa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:11.373789 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bdc7823d-5f72-4fbc-bc24-053414efbffc None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:11.379209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bdc7823d-5f72-4fbc-bc24-053414efbffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:11.422850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:11.439272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 83/166] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:11.448015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15431cdb-019b-430c-af64-29848312fa7c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:11.448131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 84/167] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:11.449436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7823d-5f72-4fbc-bc24-053414efbffc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:11.597913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7823d-5f72-4fbc-bc24-053414efbffc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:11.619726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15431cdb-019b-430c-af64-29848312fa7c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:11.632707 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:11.642295 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127b07f-42b7-4ad1-9c9c-5aa05193ec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:11.643547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127b07f-42b7-4ad1-9c9c-5aa05193ec46 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.644038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127b07f-42b7-4ad1-9c9c-5aa05193ec46 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.644603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127b07f-42b7-4ad1-9c9c-5aa05193ec46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.646191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127b07f-42b7-4ad1-9c9c-5aa05193ec46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:11.648578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127b07f-42b7-4ad1-9c9c-5aa05193ec46 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:11.650410 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 85/168] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:11.656581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127b07f-42b7-4ad1-9c9c-5aa05193ec46 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:11.675209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83949-86f0-40de-90ce-d9e7e2782bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:11.676199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83949-86f0-40de-90ce-d9e7e2782bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.676678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83949-86f0-40de-90ce-d9e7e2782bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.677145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83949-86f0-40de-90ce-d9e7e2782bf5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.688909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f83949-86f0-40de-90ce-d9e7e2782bf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-42f83949-86f0-40de-90ce-d9e7e2782bf5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:11.694415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:11.695278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:11.696080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:11.698368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 84/169] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:11.712542 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83949-86f0-40de-90ce-d9e7e2782bf5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:11.714575 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:11.773493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127b07f-42b7-4ad1-9c9c-5aa05193ec46 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:11.808739 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55ae1a-833c-4549-87a1-cb18d2ecdd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:11.809447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55ae1a-833c-4549-87a1-cb18d2ecdd20 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.809729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55ae1a-833c-4549-87a1-cb18d2ecdd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.809999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55ae1a-833c-4549-87a1-cb18d2ecdd20 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.810649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55ae1a-833c-4549-87a1-cb18d2ecdd20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:11.812093 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55ae1a-833c-4549-87a1-cb18d2ecdd20 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:11.836140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83949-86f0-40de-90ce-d9e7e2782bf5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:11.871275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ff1b3-b68d-4d06-aa60-ca0b521126f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:11.871942 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ff1b3-b68d-4d06-aa60-ca0b521126f2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:11.872227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ff1b3-b68d-4d06-aa60-ca0b521126f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:11.872502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ff1b3-b68d-4d06-aa60-ca0b521126f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:11.873167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ff1b3-b68d-4d06-aa60-ca0b521126f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:11.873548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ff1b3-b68d-4d06-aa60-ca0b521126f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:11.874951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ff1b3-b68d-4d06-aa60-ca0b521126f2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:11.883308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 85/170] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:11.884251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ff1b3-b68d-4d06-aa60-ca0b521126f2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:11.909983 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 86/171] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:11 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:43:11.911030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55ae1a-833c-4549-87a1-cb18d2ecdd20 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:11.918272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:12.003343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ff1b3-b68d-4d06-aa60-ca0b521126f2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.033107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55ae1a-833c-4549-87a1-cb18d2ecdd20 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.035948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:12.043090 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda75a60-9ced-4879-86f1-1fec89985b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.044025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda75a60-9ced-4879-86f1-1fec89985b39 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.044449 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda75a60-9ced-4879-86f1-1fec89985b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.047281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda75a60-9ced-4879-86f1-1fec89985b39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.056815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda75a60-9ced-4879-86f1-1fec89985b39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-dda75a60-9ced-4879-86f1-1fec89985b39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:12.063922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:12.064882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:12.065776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:12.070127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 86/172] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:43:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:12.070961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda75a60-9ced-4879-86f1-1fec89985b39 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:12.076170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3e94a-2364-4e73-a42a-7e8fc0667728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.076170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3e94a-2364-4e73-a42a-7e8fc0667728 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.076170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3e94a-2364-4e73-a42a-7e8fc0667728 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.076170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3e94a-2364-4e73-a42a-7e8fc0667728 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.086918 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3e94a-2364-4e73-a42a-7e8fc0667728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c3d3e94a-2364-4e73-a42a-7e8fc0667728 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:12.093201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:12.094088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:12.095263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 87/173] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:43:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:12.096138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3e94a-2364-4e73-a42a-7e8fc0667728 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:12.186354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:12.200222 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda75a60-9ced-4879-86f1-1fec89985b39 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.226649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3e94a-2364-4e73-a42a-7e8fc0667728 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.241321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920e1cb-f7d5-47ef-a2c7-61a7985e7655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.242212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920e1cb-f7d5-47ef-a2c7-61a7985e7655 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.242647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920e1cb-f7d5-47ef-a2c7-61a7985e7655 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.243085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920e1cb-f7d5-47ef-a2c7-61a7985e7655 None admin] PATH_INFO: `/v3/users/37885f237b3745c6ab47f212d423d97c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.259258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920e1cb-f7d5-47ef-a2c7-61a7985e7655 None admin] RBAC: Authorizing `identity:get_user(user_id=37885f237b3745c6ab47f212d423d97c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:12.260768 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920e1cb-f7d5-47ef-a2c7-61a7985e7655 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:12.267338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 87/174] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:12 2022] GET /identity/v3/users/37885f237b3745c6ab47f212d423d97c => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:12.267338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920e1cb-f7d5-47ef-a2c7-61a7985e7655 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:12.276754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e177e3-4052-45eb-9815-fc7ff51f6e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.277410 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e177e3-4052-45eb-9815-fc7ff51f6e90 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.277695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e177e3-4052-45eb-9815-fc7ff51f6e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.278503 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e177e3-4052-45eb-9815-fc7ff51f6e90 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.279320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e177e3-4052-45eb-9815-fc7ff51f6e90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:12.279548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e177e3-4052-45eb-9815-fc7ff51f6e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:12.280928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e177e3-4052-45eb-9815-fc7ff51f6e90 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:12.292125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 88/175] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:12.293350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e177e3-4052-45eb-9815-fc7ff51f6e90 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:12.361643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:12.387791 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920e1cb-f7d5-47ef-a2c7-61a7985e7655 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.423615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b76b7-2c4a-43c5-a4e2-79920de43f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.424292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b76b7-2c4a-43c5-a4e2-79920de43f5f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.424577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b76b7-2c4a-43c5-a4e2-79920de43f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.424852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b76b7-2c4a-43c5-a4e2-79920de43f5f None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.425554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b76b7-2c4a-43c5-a4e2-79920de43f5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:12.425867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b76b7-2c4a-43c5-a4e2-79920de43f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:12.427415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b76b7-2c4a-43c5-a4e2-79920de43f5f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:12.437271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 88/176] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:12.438945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e177e3-4052-45eb-9815-fc7ff51f6e90 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.443160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b76b7-2c4a-43c5-a4e2-79920de43f5f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:12.461596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:12.481878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4357-d8bd-472b-b19b-63b7c342f9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.482656 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4357-d8bd-472b-b19b-63b7c342f9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.482942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4357-d8bd-472b-b19b-63b7c342f9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.483371 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4357-d8bd-472b-b19b-63b7c342f9c0 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.485548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4357-d8bd-472b-b19b-63b7c342f9c0 None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:12.487020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4357-d8bd-472b-b19b-63b7c342f9c0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:12.493266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 89/177] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:12 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:12.496677 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4357-d8bd-472b-b19b-63b7c342f9c0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:12.583129 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b76b7-2c4a-43c5-a4e2-79920de43f5f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.607112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:12.618541 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4357-d8bd-472b-b19b-63b7c342f9c0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.625620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab61514-d110-48d6-8a21-a54f55b7bada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.626571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab61514-d110-48d6-8a21-a54f55b7bada None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.627050 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab61514-d110-48d6-8a21-a54f55b7bada None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.627608 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab61514-d110-48d6-8a21-a54f55b7bada None admin] PATH_INFO: `/v3/users/1a4bc4998efe45f688920bef3bc6a222` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.628992 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab61514-d110-48d6-8a21-a54f55b7bada None admin] RBAC: Authorizing `identity:get_user(user_id=1a4bc4998efe45f688920bef3bc6a222)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:12.630708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab61514-d110-48d6-8a21-a54f55b7bada None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:12.632526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 89/178] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:12 2022] GET /identity/v3/users/1a4bc4998efe45f688920bef3bc6a222 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:12.633653 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab61514-d110-48d6-8a21-a54f55b7bada None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:12.663450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d1dc-7bef-4a22-b4ef-0600f7bc4559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.664088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d1dc-7bef-4a22-b4ef-0600f7bc4559 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.664423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d1dc-7bef-4a22-b4ef-0600f7bc4559 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.664881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d1dc-7bef-4a22-b4ef-0600f7bc4559 None admin] PATH_INFO: `/v3/users/1a4bc4998efe45f688920bef3bc6a222` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.666451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3d1dc-7bef-4a22-b4ef-0600f7bc4559 None admin] RBAC: Authorizing `identity:get_user(user_id=1a4bc4998efe45f688920bef3bc6a222)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:12.668151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3d1dc-7bef-4a22-b4ef-0600f7bc4559 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:12.670967 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 90/179] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:12 2022] GET /identity/v3/users/1a4bc4998efe45f688920bef3bc6a222 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:12.676380 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d1dc-7bef-4a22-b4ef-0600f7bc4559 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:12.722897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:12.745008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab61514-d110-48d6-8a21-a54f55b7bada None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.783987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35da37-c1e3-4214-9a23-67190702b62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.784669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35da37-c1e3-4214-9a23-67190702b62b None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.784950 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35da37-c1e3-4214-9a23-67190702b62b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.785246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35da37-c1e3-4214-9a23-67190702b62b None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147/users/37885f237b3745c6ab47f212d423d97c/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.787895 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35da37-c1e3-4214-9a23-67190702b62b None admin] RBAC: Authorizing `identity:create_grant(project_id=e5580ff9cebc43c6933eb6147b391147, user_id=37885f237b3745c6ab47f212d423d97c, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:12.789683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35da37-c1e3-4214-9a23-67190702b62b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:12.809864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d1dc-7bef-4a22-b4ef-0600f7bc4559 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:12.846199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc59136-5c35-4517-9b6f-7da0a8f0567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:12.847192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc59136-5c35-4517-9b6f-7da0a8f0567b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:12.847665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc59136-5c35-4517-9b6f-7da0a8f0567b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:12.848191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc59136-5c35-4517-9b6f-7da0a8f0567b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:12.877568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 90/180] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:12 2022] PUT /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147/users/37885f237b3745c6ab47f212d423d97c/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:12.882278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35da37-c1e3-4214-9a23-67190702b62b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:12.890131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:12.983118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc59136-5c35-4517-9b6f-7da0a8f0567b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:12.984565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc59136-5c35-4517-9b6f-7da0a8f0567b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:13.043216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35da37-c1e3-4214-9a23-67190702b62b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:13.097975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d5f3d-4de6-4399-abe6-bf3cb33b1e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:13.098626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d5f3d-4de6-4399-abe6-bf3cb33b1e09 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:13.098912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d5f3d-4de6-4399-abe6-bf3cb33b1e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:13.099188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d5f3d-4de6-4399-abe6-bf3cb33b1e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:13.117551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:13.169645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 91/181] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:13.175090 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc59136-5c35-4517-9b6f-7da0a8f0567b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:13.186990 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d5f3d-4de6-4399-abe6-bf3cb33b1e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:13.188843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d5f3d-4de6-4399-abe6-bf3cb33b1e09 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:13.247808 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:13.313585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 91/182] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:13.320325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d5f3d-4de6-4399-abe6-bf3cb33b1e09 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:13.325498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc59136-5c35-4517-9b6f-7da0a8f0567b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:13.370848 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd83602-566d-4438-a095-c22c6a614ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:13.372183 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd83602-566d-4438-a095-c22c6a614ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:13.372733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd83602-566d-4438-a095-c22c6a614ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:13.373243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd83602-566d-4438-a095-c22c6a614ed4 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:13.374850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd83602-566d-4438-a095-c22c6a614ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:13.376888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd83602-566d-4438-a095-c22c6a614ed4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:13.379141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 92/183] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:13 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:13.384437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd83602-566d-4438-a095-c22c6a614ed4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:13.391866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:13.497306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd83602-566d-4438-a095-c22c6a614ed4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:13.514495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d5f3d-4de6-4399-abe6-bf3cb33b1e09 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:13.532644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:13.539540 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098eac5-4224-4ea5-8ab5-7d5951439d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:13.540457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098eac5-4224-4ea5-8ab5-7d5951439d10 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:13.540952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098eac5-4224-4ea5-8ab5-7d5951439d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:13.541463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098eac5-4224-4ea5-8ab5-7d5951439d10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:13.542743 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098eac5-4224-4ea5-8ab5-7d5951439d10 None admin] RBAC: Adding query filter params (role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.project.id=e5580ff9cebc43c6933eb6147b391147, user.id=1a4bc4998efe45f688920bef3bc6a222) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:13.545955 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098eac5-4224-4ea5-8ab5-7d5951439d10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:13.545955 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098eac5-4224-4ea5-8ab5-7d5951439d10 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:13.553035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 93/184] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:13 2022] GET /identity/v3/role_assignments?user.id=1a4bc4998efe45f688920bef3bc6a222&scope.project.id=e5580ff9cebc43c6933eb6147b391147&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:13.581703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfc9ab-e71f-4990-b6f2-8290a800bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:13.582381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfc9ab-e71f-4990-b6f2-8290a800bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:13.582661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfc9ab-e71f-4990-b6f2-8290a800bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:13.582938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfc9ab-e71f-4990-b6f2-8290a800bcc4 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:13.584169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfc9ab-e71f-4990-b6f2-8290a800bcc4 None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:13.585735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfc9ab-e71f-4990-b6f2-8290a800bcc4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:13.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 92/185] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:13 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:13.594673 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098eac5-4224-4ea5-8ab5-7d5951439d10 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:13.740412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098eac5-4224-4ea5-8ab5-7d5951439d10 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:13.781498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1348538-dcf3-4e4e-89ea-91ce21fef8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:13.782519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1348538-dcf3-4e4e-89ea-91ce21fef8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:13.783047 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1348538-dcf3-4e4e-89ea-91ce21fef8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:13.783528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1348538-dcf3-4e4e-89ea-91ce21fef8f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:13.784723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1348538-dcf3-4e4e-89ea-91ce21fef8f2 None admin] RBAC: Adding query filter params (role.id=ffd57ebcb27d45ef9b9d642f409cc6d7, scope.project.id=e5580ff9cebc43c6933eb6147b391147, user.id=1a4bc4998efe45f688920bef3bc6a222) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:13.785257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1348538-dcf3-4e4e-89ea-91ce21fef8f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:13.787001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1348538-dcf3-4e4e-89ea-91ce21fef8f2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:13.794295 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 94/186] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:13 2022] GET /identity/v3/role_assignments?user.id=1a4bc4998efe45f688920bef3bc6a222&scope.project.id=e5580ff9cebc43c6933eb6147b391147&role.id=ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:14.352798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c376-5632-4efc-b3b3-4bb1ec4d02cf None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:14.353389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c376-5632-4efc-b3b3-4bb1ec4d02cf None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:14.353389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c376-5632-4efc-b3b3-4bb1ec4d02cf None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:14.354721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 93/187] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:14.360491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf6380-d027-41ef-a7bd-7321d925bd30 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:14.360912 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf6380-d027-41ef-a7bd-7321d925bd30 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:14.361095 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf6380-d027-41ef-a7bd-7321d925bd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:14.380542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1adf6380-d027-41ef-a7bd-7321d925bd30 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:14.381723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1adf6380-d027-41ef-a7bd-7321d925bd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:14.413166 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cfd8c-16df-4c2a-9491-cc493f5f2a46 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:14.413476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cfd8c-16df-4c2a-9491-cc493f5f2a46 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:14.413755 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cfd8c-16df-4c2a-9491-cc493f5f2a46 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:14.415111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 94/188] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:14.420196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99c944-0d22-4489-b202-eb9ed4174eab None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:14.420475 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99c944-0d22-4489-b202-eb9ed4174eab None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:14.420748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99c944-0d22-4489-b202-eb9ed4174eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:14.438219 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:14.438978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8b99c944-0d22-4489-b202-eb9ed4174eab None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:14.440166 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8b99c944-0d22-4489-b202-eb9ed4174eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:14.453653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 95/189] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:14.509242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 95/190] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:14.728180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf6380-d027-41ef-a7bd-7321d925bd30 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:14.774469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99c944-0d22-4489-b202-eb9ed4174eab None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:14.809512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:14.834189 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf6380-d027-41ef-a7bd-7321d925bd30 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:14.870378 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161e3a9-00d9-4cd6-ab6a-43f4c4610765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:14.871072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161e3a9-00d9-4cd6-ab6a-43f4c4610765 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:14.871357 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161e3a9-00d9-4cd6-ab6a-43f4c4610765 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:14.872316 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161e3a9-00d9-4cd6-ab6a-43f4c4610765 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:14.873368 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161e3a9-00d9-4cd6-ab6a-43f4c4610765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:14.874808 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161e3a9-00d9-4cd6-ab6a-43f4c4610765 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:14.876530 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 96/191] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:14.876857 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99c944-0d22-4489-b202-eb9ed4174eab None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:14.881670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161e3a9-00d9-4cd6-ab6a-43f4c4610765 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:14.911137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28369021-aa55-44e8-b4f7-90dd49bf05e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:14.911771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28369021-aa55-44e8-b4f7-90dd49bf05e9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:14.912052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28369021-aa55-44e8-b4f7-90dd49bf05e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:14.912329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28369021-aa55-44e8-b4f7-90dd49bf05e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:14.926081 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28369021-aa55-44e8-b4f7-90dd49bf05e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-28369021-aa55-44e8-b4f7-90dd49bf05e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:14.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:14.931851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:14.932675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 96/192] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:14.938734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28369021-aa55-44e8-b4f7-90dd49bf05e9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:14.945275 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:14.992806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161e3a9-00d9-4cd6-ab6a-43f4c4610765 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:15.029744 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a09e3-5363-4bb6-a4b3-a80ba96d4035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:15.030360 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a09e3-5363-4bb6-a4b3-a80ba96d4035 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:15.030641 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a09e3-5363-4bb6-a4b3-a80ba96d4035 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:15.030914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a09e3-5363-4bb6-a4b3-a80ba96d4035 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:15.031569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a09e3-5363-4bb6-a4b3-a80ba96d4035 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:15.033044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a09e3-5363-4bb6-a4b3-a80ba96d4035 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:15.041200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:15.048931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28369021-aa55-44e8-b4f7-90dd49bf05e9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:15.083519 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f74c2-50ce-47dc-940c-b3d5e492df53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:15.084136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f74c2-50ce-47dc-940c-b3d5e492df53 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:15.084424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f74c2-50ce-47dc-940c-b3d5e492df53 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:15.084702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f74c2-50ce-47dc-940c-b3d5e492df53 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:15.085450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f74c2-50ce-47dc-940c-b3d5e492df53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:15.085759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f74c2-50ce-47dc-940c-b3d5e492df53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:15.087153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f74c2-50ce-47dc-940c-b3d5e492df53 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:15.095994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 97/193] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:15.101547 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f74c2-50ce-47dc-940c-b3d5e492df53 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:15.121446 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 97/194] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:14 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:43:15.212227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f74c2-50ce-47dc-940c-b3d5e492df53 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:15.248878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca17b2c-677c-4d11-8962-2ae1b1246779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:15.249486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca17b2c-677c-4d11-8962-2ae1b1246779 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:15.249765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca17b2c-677c-4d11-8962-2ae1b1246779 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:15.250035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca17b2c-677c-4d11-8962-2ae1b1246779 None admin] PATH_INFO: `/v3/users/08115692afb14d17951a919a65820ea2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:15.265547 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca17b2c-677c-4d11-8962-2ae1b1246779 None admin] RBAC: Authorizing `identity:get_user(user_id=08115692afb14d17951a919a65820ea2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:15.266987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca17b2c-677c-4d11-8962-2ae1b1246779 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:15.268912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 98/195] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:15 2022] GET /identity/v3/users/08115692afb14d17951a919a65820ea2 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:15.274343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a09e3-5363-4bb6-a4b3-a80ba96d4035 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:15.390684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a09e3-5363-4bb6-a4b3-a80ba96d4035 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:15.404981 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:15.429520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b23320-f6ca-4c03-b9d5-10f38833c29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:15.430784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23320-f6ca-4c03-b9d5-10f38833c29d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:15.430784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23320-f6ca-4c03-b9d5-10f38833c29d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:15.430784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23320-f6ca-4c03-b9d5-10f38833c29d None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:15.432543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b23320-f6ca-4c03-b9d5-10f38833c29d None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:15.434404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b23320-f6ca-4c03-b9d5-10f38833c29d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:15.436020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 98/196] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:15 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:15.441761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca17b2c-677c-4d11-8962-2ae1b1246779 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:15.450491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02d0e3-243e-4243-88b4-06af446be382 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:15.450806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02d0e3-243e-4243-88b4-06af446be382 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:15.453541 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02d0e3-243e-4243-88b4-06af446be382 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:15.453541 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 99/197] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:43:15.458296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d090-ba9d-47f0-b1f0-62d221e4dfa5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:15.458583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d090-ba9d-47f0-b1f0-62d221e4dfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:15.458909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d090-ba9d-47f0-b1f0-62d221e4dfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:15.464512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:15.478806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d441d090-ba9d-47f0-b1f0-62d221e4dfa5 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:15.480134 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d441d090-ba9d-47f0-b1f0-62d221e4dfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:15.573220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 100/198] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:15.578038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca17b2c-677c-4d11-8962-2ae1b1246779 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:15.613665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47299c08-c622-4071-9917-9537311934c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:15.614334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47299c08-c622-4071-9917-9537311934c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:15.614616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47299c08-c622-4071-9917-9537311934c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:15.614892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47299c08-c622-4071-9917-9537311934c6 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147/users/08115692afb14d17951a919a65820ea2/roles/8b0de894383f4c18a3363436c36841fa` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:15.616831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47299c08-c622-4071-9917-9537311934c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5580ff9cebc43c6933eb6147b391147, user_id=08115692afb14d17951a919a65820ea2, role_id=8b0de894383f4c18a3363436c36841fa)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:15.618380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47299c08-c622-4071-9917-9537311934c6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:15.715615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 99/199] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:15 2022] PUT /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147/users/08115692afb14d17951a919a65820ea2/roles/8b0de894383f4c18a3363436c36841fa => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:15.855014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441d090-ba9d-47f0-b1f0-62d221e4dfa5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:15.962732 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441d090-ba9d-47f0-b1f0-62d221e4dfa5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:15.974849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:16.000128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45884aad-9b42-4b79-bf68-a39e5faa08a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:16.001128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45884aad-9b42-4b79-bf68-a39e5faa08a3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:16.001636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45884aad-9b42-4b79-bf68-a39e5faa08a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:16.002209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45884aad-9b42-4b79-bf68-a39e5faa08a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:16.014390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45884aad-9b42-4b79-bf68-a39e5faa08a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-45884aad-9b42-4b79-bf68-a39e5faa08a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:16.019705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:16.020572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:16.021401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:16.024080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 101/200] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:16.030816 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47299c08-c622-4071-9917-9537311934c6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:16.115202 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:16.139678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47299c08-c622-4071-9917-9537311934c6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:16.179169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e63b1-e956-43c3-bfcc-2c4c773b8662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:16.180167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e63b1-e956-43c3-bfcc-2c4c773b8662 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:16.180693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e63b1-e956-43c3-bfcc-2c4c773b8662 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:16.181168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e63b1-e956-43c3-bfcc-2c4c773b8662 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:16.182098 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e63b1-e956-43c3-bfcc-2c4c773b8662 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:16.182612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e63b1-e956-43c3-bfcc-2c4c773b8662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:16.184787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e63b1-e956-43c3-bfcc-2c4c773b8662 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:16.194776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 100/201] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:16.200756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45884aad-9b42-4b79-bf68-a39e5faa08a3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:16.309551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45884aad-9b42-4b79-bf68-a39e5faa08a3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:16.344428 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9998ea-d5f8-47ee-ae04-505ced69a94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:16.345393 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9998ea-d5f8-47ee-ae04-505ced69a94e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:16.345881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9998ea-d5f8-47ee-ae04-505ced69a94e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:16.346363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9998ea-d5f8-47ee-ae04-505ced69a94e None admin] PATH_INFO: `/v3/users/37885f237b3745c6ab47f212d423d97c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:16.347724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9998ea-d5f8-47ee-ae04-505ced69a94e None admin] RBAC: Authorizing `identity:get_user(user_id=37885f237b3745c6ab47f212d423d97c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:16.349478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9998ea-d5f8-47ee-ae04-505ced69a94e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:16.351646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 102/202] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:16 2022] GET /identity/v3/users/37885f237b3745c6ab47f212d423d97c => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:16.358544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e63b1-e956-43c3-bfcc-2c4c773b8662 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:16.468875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e63b1-e956-43c3-bfcc-2c4c773b8662 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:16.505539 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8c283-0510-4553-97b8-722d532bc78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:16.506529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8c283-0510-4553-97b8-722d532bc78b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:16.507014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8c283-0510-4553-97b8-722d532bc78b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:16.507488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8c283-0510-4553-97b8-722d532bc78b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:16.518030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:16.586058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8c283-0510-4553-97b8-722d532bc78b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:16.587616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8c283-0510-4553-97b8-722d532bc78b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:16.692770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 101/203] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:16.698411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9998ea-d5f8-47ee-ae04-505ced69a94e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:16.745774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:16.813162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9998ea-d5f8-47ee-ae04-505ced69a94e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:16.848600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6268f-8c2a-467d-b13d-06f580ecc8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:16.849573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6268f-8c2a-467d-b13d-06f580ecc8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:16.850056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6268f-8c2a-467d-b13d-06f580ecc8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:16.850550 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6268f-8c2a-467d-b13d-06f580ecc8d2 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:16.851891 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6268f-8c2a-467d-b13d-06f580ecc8d2 None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:16.853626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6268f-8c2a-467d-b13d-06f580ecc8d2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:16.855500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 103/204] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:16 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:16.861607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8c283-0510-4553-97b8-722d532bc78b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:16.902038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:16.965349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8c283-0510-4553-97b8-722d532bc78b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:17.001686 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7818e-d67e-4fc8-99ac-7404f92b46bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:17.002221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7818e-d67e-4fc8-99ac-7404f92b46bb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:17.002503 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7818e-d67e-4fc8-99ac-7404f92b46bb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:17.002782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7818e-d67e-4fc8-99ac-7404f92b46bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:17.003520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7818e-d67e-4fc8-99ac-7404f92b46bb None admin] RBAC: Adding query filter params (role.id=f1fda4fad003483dbc5843e16b6dc9d2, scope.project.id=e5580ff9cebc43c6933eb6147b391147, user.id=37885f237b3745c6ab47f212d423d97c) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:17.003832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7818e-d67e-4fc8-99ac-7404f92b46bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:17.005309 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7818e-d67e-4fc8-99ac-7404f92b46bb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:17.011750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 102/205] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:16 2022] GET /identity/v3/role_assignments?user.id=37885f237b3745c6ab47f212d423d97c&scope.project.id=e5580ff9cebc43c6933eb6147b391147&role.id=f1fda4fad003483dbc5843e16b6dc9d2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:17.461923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f44559-d68d-472f-b4a7-affa457ad117 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:17.462345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f44559-d68d-472f-b4a7-affa457ad117 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:17.462526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f44559-d68d-472f-b4a7-affa457ad117 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:17.464081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 104/206] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:17.470462 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1101c8-9f92-4f38-9de4-4591f4cad3b3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:17.470971 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1101c8-9f92-4f38-9de4-4591f4cad3b3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:17.471081 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1101c8-9f92-4f38-9de4-4591f4cad3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:17.490554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6d1101c8-9f92-4f38-9de4-4591f4cad3b3 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:17.491749 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6d1101c8-9f92-4f38-9de4-4591f4cad3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:17.568393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 103/207] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:17.579365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ffefb-1b77-4609-93b0-787c673799fe None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:17.579652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ffefb-1b77-4609-93b0-787c673799fe None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:17.579932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ffefb-1b77-4609-93b0-787c673799fe None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:17.581566 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 105/208] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:17.586489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577f201-e901-43c1-bcf4-f2aef3af4e25 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:17.586789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577f201-e901-43c1-bcf4-f2aef3af4e25 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:17.587067 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577f201-e901-43c1-bcf4-f2aef3af4e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:17.605310 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9577f201-e901-43c1-bcf4-f2aef3af4e25 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:17.606787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9577f201-e901-43c1-bcf4-f2aef3af4e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:17.633657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:17.675322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 104/209] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:17.836161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ffefb-1b77-4609-93b0-787c673799fe None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:17.940372 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ffefb-1b77-4609-93b0-787c673799fe None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:17.948351 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:17.951245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577f201-e901-43c1-bcf4-f2aef3af4e25 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:17.977515 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c801e48-fcaa-4642-a765-a845830a5952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:17.978699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c801e48-fcaa-4642-a765-a845830a5952 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:17.979452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c801e48-fcaa-4642-a765-a845830a5952 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:17.979723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c801e48-fcaa-4642-a765-a845830a5952 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:17.991201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c801e48-fcaa-4642-a765-a845830a5952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1c801e48-fcaa-4642-a765-a845830a5952 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:17.997270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:17.998240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:17.999029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:18.002936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 106/210] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:18.004143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c801e48-fcaa-4642-a765-a845830a5952 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:18.058964 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577f201-e901-43c1-bcf4-f2aef3af4e25 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:18.092954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b782be-654a-47c3-9383-d08ce031a1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:18.093551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b782be-654a-47c3-9383-d08ce031a1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:18.093836 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b782be-654a-47c3-9383-d08ce031a1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:18.094113 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b782be-654a-47c3-9383-d08ce031a1a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:18.095214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b782be-654a-47c3-9383-d08ce031a1a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:18.096685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b782be-654a-47c3-9383-d08ce031a1a2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:18.098303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 105/211] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:18.099425 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385acb2-d597-49f1-a244-0d4f2f4ca8ea None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:18.099710 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385acb2-d597-49f1-a244-0d4f2f4ca8ea None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:18.099988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385acb2-d597-49f1-a244-0d4f2f4ca8ea None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:18.101213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 106/212] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:18.103503 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385acb2-d597-49f1-a244-0d4f2f4ca8ea None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:18.112260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c801e48-fcaa-4642-a765-a845830a5952 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:18.134008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:18.145482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dbbf3-d762-4b46-a86e-fa67d86656f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:18.146091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dbbf3-d762-4b46-a86e-fa67d86656f7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:18.149375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dbbf3-d762-4b46-a86e-fa67d86656f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:18.149375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dbbf3-d762-4b46-a86e-fa67d86656f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:18.149375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dbbf3-d762-4b46-a86e-fa67d86656f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:18.149375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dbbf3-d762-4b46-a86e-fa67d86656f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:18.149375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dbbf3-d762-4b46-a86e-fa67d86656f7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:18.157459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 107/213] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:18.158892 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04610c24-25c6-4133-be65-033975f91aa1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:18.159178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04610c24-25c6-4133-be65-033975f91aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:18.159455 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04610c24-25c6-4133-be65-033975f91aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:18.177947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-04610c24-25c6-4133-be65-033975f91aa1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:18.179257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-04610c24-25c6-4133-be65-033975f91aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:18.201738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:18.205181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385acb2-d597-49f1-a244-0d4f2f4ca8ea None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:18.238847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bca7e-e631-4103-ba61-4ce067cb98fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:18.239455 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bca7e-e631-4103-ba61-4ce067cb98fe None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:18.239737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bca7e-e631-4103-ba61-4ce067cb98fe None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:18.240011 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bca7e-e631-4103-ba61-4ce067cb98fe None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:18.240659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bca7e-e631-4103-ba61-4ce067cb98fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:18.242128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bca7e-e631-4103-ba61-4ce067cb98fe None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:18.245330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 108/214] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:18.246165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04610c24-25c6-4133-be65-033975f91aa1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:18.349106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 107/215] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:18 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:18.359726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04610c24-25c6-4133-be65-033975f91aa1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:18.403363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05678156-c0e7-4e3e-82b0-973bf9dc3c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:18.404102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678156-c0e7-4e3e-82b0-973bf9dc3c21 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:18.404391 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678156-c0e7-4e3e-82b0-973bf9dc3c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:18.404670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678156-c0e7-4e3e-82b0-973bf9dc3c21 None admin] PATH_INFO: `/v3/users/841dc4d48b7448b5801b69f695649a84` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:18.421080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05678156-c0e7-4e3e-82b0-973bf9dc3c21 None admin] RBAC: Authorizing `identity:get_user(user_id=841dc4d48b7448b5801b69f695649a84)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:18.423582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05678156-c0e7-4e3e-82b0-973bf9dc3c21 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:18.426695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 109/216] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:18 2022] GET /identity/v3/users/841dc4d48b7448b5801b69f695649a84 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:18.433049 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bca7e-e631-4103-ba61-4ce067cb98fe None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:18.524746 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05678156-c0e7-4e3e-82b0-973bf9dc3c21 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:18.528673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:18.545669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bca7e-e631-4103-ba61-4ce067cb98fe None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:18.550549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:18.584599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58a646-387d-4bfe-b1f0-396b3196d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:18.585236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a646-387d-4bfe-b1f0-396b3196d295 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:18.585524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a646-387d-4bfe-b1f0-396b3196d295 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:18.585795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a646-387d-4bfe-b1f0-396b3196d295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:18.586526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58a646-387d-4bfe-b1f0-396b3196d295 None admin] RBAC: Adding query filter params (role.id=f1fda4fad003483dbc5843e16b6dc9d2, user.id=841dc4d48b7448b5801b69f695649a84, scope.system=all) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:18.586837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58a646-387d-4bfe-b1f0-396b3196d295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:18.588292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58a646-387d-4bfe-b1f0-396b3196d295 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:18.598769 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 108/217] 10.222.0.30 () {56 vars in 1308 bytes} [Sat Oct 15 12:43:18 2022] GET /identity/v3/role_assignments?user.id=841dc4d48b7448b5801b69f695649a84&scope.system=all&role.id=f1fda4fad003483dbc5843e16b6dc9d2 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:18.654951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05678156-c0e7-4e3e-82b0-973bf9dc3c21 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:18.697665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59345b1-675d-44eb-a394-076c76416edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:18.698325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59345b1-675d-44eb-a394-076c76416edb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:18.698609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59345b1-675d-44eb-a394-076c76416edb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:18.698883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59345b1-675d-44eb-a394-076c76416edb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:18.711839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59345b1-675d-44eb-a394-076c76416edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f59345b1-675d-44eb-a394-076c76416edb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:18.717126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:18.718138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:18.718957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 110/218] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:18.725884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58a646-387d-4bfe-b1f0-396b3196d295 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:18.839161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58a646-387d-4bfe-b1f0-396b3196d295 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:18.875799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc52ad7-8dc0-4a22-96ad-e76a0aecc771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:18.876420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc52ad7-8dc0-4a22-96ad-e76a0aecc771 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:18.876697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc52ad7-8dc0-4a22-96ad-e76a0aecc771 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:18.876973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc52ad7-8dc0-4a22-96ad-e76a0aecc771 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:18.877677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc52ad7-8dc0-4a22-96ad-e76a0aecc771 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:18.877984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc52ad7-8dc0-4a22-96ad-e76a0aecc771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:18.879385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc52ad7-8dc0-4a22-96ad-e76a0aecc771 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:18.888872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 109/219] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:18.894137 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59345b1-675d-44eb-a394-076c76416edb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:18.973982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:18.998101 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59345b1-675d-44eb-a394-076c76416edb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:19.034170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d9b32-0f00-456a-9022-bfbcfb6f06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:19.034809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d9b32-0f00-456a-9022-bfbcfb6f06a4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:19.035089 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d9b32-0f00-456a-9022-bfbcfb6f06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:19.035361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d9b32-0f00-456a-9022-bfbcfb6f06a4 None admin] PATH_INFO: `/v3/users/08115692afb14d17951a919a65820ea2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:19.036496 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d9b32-0f00-456a-9022-bfbcfb6f06a4 None admin] RBAC: Authorizing `identity:get_user(user_id=08115692afb14d17951a919a65820ea2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:19.037973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d9b32-0f00-456a-9022-bfbcfb6f06a4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:19.039733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 111/220] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:18 2022] GET /identity/v3/users/08115692afb14d17951a919a65820ea2 => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:19.045587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc52ad7-8dc0-4a22-96ad-e76a0aecc771 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:19.061855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:19.152034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc52ad7-8dc0-4a22-96ad-e76a0aecc771 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:19.187370 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5d5ee-32ad-459c-ac06-0763f5200dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:19.187992 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5d5ee-32ad-459c-ac06-0763f5200dca None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:19.188273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5d5ee-32ad-459c-ac06-0763f5200dca None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:19.188546 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5d5ee-32ad-459c-ac06-0763f5200dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:19.257490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5d5ee-32ad-459c-ac06-0763f5200dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:19.258911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5d5ee-32ad-459c-ac06-0763f5200dca None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:19.273525 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:19.363456 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 110/221] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:19.369172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d9b32-0f00-456a-9022-bfbcfb6f06a4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:19.476921 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d9b32-0f00-456a-9022-bfbcfb6f06a4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:19.515129 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d16d1f-9a5e-4716-8b2a-267635d93357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:19.515694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d16d1f-9a5e-4716-8b2a-267635d93357 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:19.515972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d16d1f-9a5e-4716-8b2a-267635d93357 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:19.516246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d16d1f-9a5e-4716-8b2a-267635d93357 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:19.517408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d16d1f-9a5e-4716-8b2a-267635d93357 None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:19.518848 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d16d1f-9a5e-4716-8b2a-267635d93357 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:19.520826 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 112/222] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:19 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:19.526235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5d5ee-32ad-459c-ac06-0763f5200dca None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:19.647415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5d5ee-32ad-459c-ac06-0763f5200dca None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:19.661543 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:19.683164 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e4455-1843-42c4-b4ee-2eb04484fbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:19.694236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e4455-1843-42c4-b4ee-2eb04484fbdd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:19.694236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e4455-1843-42c4-b4ee-2eb04484fbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:19.694236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e4455-1843-42c4-b4ee-2eb04484fbdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:19.694236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e4455-1843-42c4-b4ee-2eb04484fbdd None admin] RBAC: Adding query filter params (role.id=8b0de894383f4c18a3363436c36841fa, scope.project.id=e5580ff9cebc43c6933eb6147b391147, user.id=08115692afb14d17951a919a65820ea2) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:19.694236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e4455-1843-42c4-b4ee-2eb04484fbdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:19.694236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e4455-1843-42c4-b4ee-2eb04484fbdd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:19.694713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 111/223] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:19 2022] GET /identity/v3/role_assignments?user.id=08115692afb14d17951a919a65820ea2&scope.project.id=e5580ff9cebc43c6933eb6147b391147&role.id=8b0de894383f4c18a3363436c36841fa => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:20.760129 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57f76-656a-4749-86cc-f5cf2ddc75d7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:20.760632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57f76-656a-4749-86cc-f5cf2ddc75d7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:20.760705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57f76-656a-4749-86cc-f5cf2ddc75d7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:20.763184 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 113/224] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:20.768457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68b9aa-f743-44a4-ae1f-8ea4ad1dca5b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:20.768965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68b9aa-f743-44a4-ae1f-8ea4ad1dca5b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:20.769166 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68b9aa-f743-44a4-ae1f-8ea4ad1dca5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:20.781351 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf82b47-6ffa-40c7-a401-742eb7059850 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:20.781691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf82b47-6ffa-40c7-a401-742eb7059850 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:20.781912 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf82b47-6ffa-40c7-a401-742eb7059850 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:20.783501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 114/225] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:43:20.789598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79df586-4456-4a21-80a0-cc81ba19744f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:20.789897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79df586-4456-4a21-80a0-cc81ba19744f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:20.790154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79df586-4456-4a21-80a0-cc81ba19744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:20.790359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9d68b9aa-f743-44a4-ae1f-8ea4ad1dca5b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:20.791624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9d68b9aa-f743-44a4-ae1f-8ea4ad1dca5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:20.809775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c79df586-4456-4a21-80a0-cc81ba19744f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:20.810893 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c79df586-4456-4a21-80a0-cc81ba19744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:20.819329 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:20.862344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 112/226] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:20.880423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 115/227] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:20.893018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f399158-388e-450f-a3e5-c3f1b4fd9675 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:20.893403 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f399158-388e-450f-a3e5-c3f1b4fd9675 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:20.893745 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f399158-388e-450f-a3e5-c3f1b4fd9675 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:20.895189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 113/228] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:20.900313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52de245-3209-48a7-b986-4a923dcf1f1a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:20.900599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52de245-3209-48a7-b986-4a923dcf1f1a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:20.900872 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52de245-3209-48a7-b986-4a923dcf1f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:20.919921 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f52de245-3209-48a7-b986-4a923dcf1f1a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:20.921215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f52de245-3209-48a7-b986-4a923dcf1f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:20.989463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 116/229] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:21.131222 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f399158-388e-450f-a3e5-c3f1b4fd9675 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:21.147435 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52de245-3209-48a7-b986-4a923dcf1f1a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:21.221817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:21.238685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f399158-388e-450f-a3e5-c3f1b4fd9675 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:21.247637 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:21.253095 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52de245-3209-48a7-b986-4a923dcf1f1a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:21.273362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507cf0b-d0e3-4189-969b-c1a97c675c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:21.273999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507cf0b-d0e3-4189-969b-c1a97c675c59 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:21.274276 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507cf0b-d0e3-4189-969b-c1a97c675c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:21.274546 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507cf0b-d0e3-4189-969b-c1a97c675c59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:21.286077 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507cf0b-d0e3-4189-969b-c1a97c675c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:21.288830 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045793c3-2ba5-4466-a5b1-952e5e337bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:21.289503 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045793c3-2ba5-4466-a5b1-952e5e337bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:21.289786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045793c3-2ba5-4466-a5b1-952e5e337bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:21.293988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045793c3-2ba5-4466-a5b1-952e5e337bc2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c507cf0b-d0e3-4189-969b-c1a97c675c59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:21.294084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:21.294930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:21.295722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:21.296320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045793c3-2ba5-4466-a5b1-952e5e337bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:21.296394 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 114/230] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:21.296467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045793c3-2ba5-4466-a5b1-952e5e337bc2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:21.296534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507cf0b-d0e3-4189-969b-c1a97c675c59 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:21.296598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 117/231] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:21.298664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045793c3-2ba5-4466-a5b1-952e5e337bc2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:21.393641 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507cf0b-d0e3-4189-969b-c1a97c675c59 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:21.396079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045793c3-2ba5-4466-a5b1-952e5e337bc2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:21.427446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a9350-67b4-4a73-ac9a-940157aa9025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:21.428071 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9350-67b4-4a73-ac9a-940157aa9025 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:21.428355 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9350-67b4-4a73-ac9a-940157aa9025 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:21.428630 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9350-67b4-4a73-ac9a-940157aa9025 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:21.430410 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dfdd4-fcc7-4967-82c5-229c96bd5cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:21.431022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dfdd4-fcc7-4967-82c5-229c96bd5cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:21.431300 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dfdd4-fcc7-4967-82c5-229c96bd5cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:21.431575 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dfdd4-fcc7-4967-82c5-229c96bd5cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:21.432234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dfdd4-fcc7-4967-82c5-229c96bd5cb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:21.432541 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dfdd4-fcc7-4967-82c5-229c96bd5cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:21.434902 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dfdd4-fcc7-4967-82c5-229c96bd5cb0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:21.439182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a9350-67b4-4a73-ac9a-940157aa9025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:21.442871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 118/232] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:21.443639 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dfdd4-fcc7-4967-82c5-229c96bd5cb0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-394a9350-67b4-4a73-ac9a-940157aa9025 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:21.443993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:21.444855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:21.445704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:21.448085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:21.450115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 115/233] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:21.451096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a9350-67b4-4a73-ac9a-940157aa9025 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:21.479399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:21.546643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dfdd4-fcc7-4967-82c5-229c96bd5cb0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:21.550331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a9350-67b4-4a73-ac9a-940157aa9025 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:21.581466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675aa75-95bf-478e-9327-ff7e72a037f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:21.582080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675aa75-95bf-478e-9327-ff7e72a037f3 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:21.582359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675aa75-95bf-478e-9327-ff7e72a037f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:21.582632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675aa75-95bf-478e-9327-ff7e72a037f3 None admin] PATH_INFO: `/v3/groups` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:21.583259 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675aa75-95bf-478e-9327-ff7e72a037f3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:21.584696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675aa75-95bf-478e-9327-ff7e72a037f3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:21.585266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bd5e6-6cf5-4fbe-a2fa-b109f48222ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:21.585906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bd5e6-6cf5-4fbe-a2fa-b109f48222ed None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:21.586186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bd5e6-6cf5-4fbe-a2fa-b109f48222ed None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:21.586461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bd5e6-6cf5-4fbe-a2fa-b109f48222ed None admin] PATH_INFO: `/v3/users/3d454bafc80f4af1a71aa4549424cac8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:21.599502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bd5e6-6cf5-4fbe-a2fa-b109f48222ed None admin] RBAC: Authorizing `identity:get_user(user_id=3d454bafc80f4af1a71aa4549424cac8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:21.600979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bd5e6-6cf5-4fbe-a2fa-b109f48222ed None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:21.602631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 116/234] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:21 2022] GET /identity/v3/users/3d454bafc80f4af1a71aa4549424cac8 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:21.603395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bd5e6-6cf5-4fbe-a2fa-b109f48222ed None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:21.662967 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 119/235] 10.222.0.30 () {60 vars in 1134 bytes} [Sat Oct 15 12:43:21 2022] POST /identity/v3/groups => generated 229 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:21.663718 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675aa75-95bf-478e-9327-ff7e72a037f3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:21.689431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:21.706611 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bd5e6-6cf5-4fbe-a2fa-b109f48222ed None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:21.748608 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba5a0f-5571-4cc6-85f7-a4cc2cc79273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:21.749267 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba5a0f-5571-4cc6-85f7-a4cc2cc79273 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:21.749547 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba5a0f-5571-4cc6-85f7-a4cc2cc79273 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:21.749823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba5a0f-5571-4cc6-85f7-a4cc2cc79273 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:21.750506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba5a0f-5571-4cc6-85f7-a4cc2cc79273 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:21.750807 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba5a0f-5571-4cc6-85f7-a4cc2cc79273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:21.752231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba5a0f-5571-4cc6-85f7-a4cc2cc79273 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:21.752694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:21.761697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 117/236] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:21.767668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba5a0f-5571-4cc6-85f7-a4cc2cc79273 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:21.800567 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675aa75-95bf-478e-9327-ff7e72a037f3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:21.843143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca593f-1dd0-40ec-824f-18d7cad0a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:21.843770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca593f-1dd0-40ec-824f-18d7cad0a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:21.844051 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca593f-1dd0-40ec-824f-18d7cad0a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:21.844329 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca593f-1dd0-40ec-824f-18d7cad0a6d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:21.845069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca593f-1dd0-40ec-824f-18d7cad0a6d7 None admin] RBAC: Adding query filter params (role.id=8b0de894383f4c18a3363436c36841fa, user.id=3d454bafc80f4af1a71aa4549424cac8, scope.system=all) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:21.845417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca593f-1dd0-40ec-824f-18d7cad0a6d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:21.846850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca593f-1dd0-40ec-824f-18d7cad0a6d7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:21.852312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 120/237] 10.222.0.30 () {56 vars in 1308 bytes} [Sat Oct 15 12:43:21 2022] GET /identity/v3/role_assignments?user.id=3d454bafc80f4af1a71aa4549424cac8&scope.system=all&role.id=8b0de894383f4c18a3363436c36841fa => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:21.884635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba5a0f-5571-4cc6-85f7-a4cc2cc79273 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:21.925595 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eea4f0-38cd-4be0-b36e-9d6c9957ee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:21.926177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea4f0-38cd-4be0-b36e-9d6c9957ee53 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:21.926461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea4f0-38cd-4be0-b36e-9d6c9957ee53 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:21.926740 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea4f0-38cd-4be0-b36e-9d6c9957ee53 None admin] PATH_INFO: `/v3/users/841dc4d48b7448b5801b69f695649a84` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:21.927954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eea4f0-38cd-4be0-b36e-9d6c9957ee53 None admin] RBAC: Authorizing `identity:get_user(user_id=841dc4d48b7448b5801b69f695649a84)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:21.929490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eea4f0-38cd-4be0-b36e-9d6c9957ee53 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:21.931361 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 118/238] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:21 2022] GET /identity/v3/users/841dc4d48b7448b5801b69f695649a84 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:21.938345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca593f-1dd0-40ec-824f-18d7cad0a6d7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:22.054013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca593f-1dd0-40ec-824f-18d7cad0a6d7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:22.079809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:22.088710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763a1f4-2847-4372-8dab-1bb86d111b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:22.089407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763a1f4-2847-4372-8dab-1bb86d111b86 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:22.089690 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763a1f4-2847-4372-8dab-1bb86d111b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:22.090487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763a1f4-2847-4372-8dab-1bb86d111b86 None admin] PATH_INFO: `/v3/system/users/841dc4d48b7448b5801b69f695649a84/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:22.091942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763a1f4-2847-4372-8dab-1bb86d111b86 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=841dc4d48b7448b5801b69f695649a84, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:22.093367 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763a1f4-2847-4372-8dab-1bb86d111b86 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:22.166621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 121/239] 10.222.0.30 () {58 vars in 1334 bytes} [Sat Oct 15 12:43:21 2022] PUT /identity/v3/system/users/841dc4d48b7448b5801b69f695649a84/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:23.866328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba2889-3002-45c5-87a3-ae3b10dee8ea None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:23.866871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba2889-3002-45c5-87a3-ae3b10dee8ea None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:23.866871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba2889-3002-45c5-87a3-ae3b10dee8ea None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:23.868284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 119/240] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:23.873421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158869c-06ff-4241-a20e-485690ad612b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:23.873849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158869c-06ff-4241-a20e-485690ad612b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:23.874017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158869c-06ff-4241-a20e-485690ad612b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:23.894922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5158869c-06ff-4241-a20e-485690ad612b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:23.896163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5158869c-06ff-4241-a20e-485690ad612b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:23.968158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 122/241] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:24.014144 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88c4fc-4765-4e25-a28f-fad2990ef5bb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.014438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88c4fc-4765-4e25-a28f-fad2990ef5bb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.014702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88c4fc-4765-4e25-a28f-fad2990ef5bb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.016205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 120/242] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:24.021128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394f24-82b5-47ab-98db-bcd6209dc52b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.021465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394f24-82b5-47ab-98db-bcd6209dc52b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.021748 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394f24-82b5-47ab-98db-bcd6209dc52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.040619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7b394f24-82b5-47ab-98db-bcd6209dc52b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:24.041845 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7b394f24-82b5-47ab-98db-bcd6209dc52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:24.110304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 123/243] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:24.234162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88c4fc-4765-4e25-a28f-fad2990ef5bb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:24.250902 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:24.324195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a72f-74ef-491c-a2e8-79eba40a5068 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.325131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a72f-74ef-491c-a2e8-79eba40a5068 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.325371 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a72f-74ef-491c-a2e8-79eba40a5068 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.328025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 124/244] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:43:24.332022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8b642-1de2-490c-afc0-a4cf72c966d6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.332313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8b642-1de2-490c-afc0-a4cf72c966d6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.332596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8b642-1de2-490c-afc0-a4cf72c966d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.341387 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88c4fc-4765-4e25-a28f-fad2990ef5bb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:24.350459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fbe8b642-1de2-490c-afc0-a4cf72c966d6 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:24.351649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fbe8b642-1de2-490c-afc0-a4cf72c966d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:24.377694 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc8612-d236-41fa-a6d4-3bd5598f7ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:24.378421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc8612-d236-41fa-a6d4-3bd5598f7ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.378703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc8612-d236-41fa-a6d4-3bd5598f7ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.378977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc8612-d236-41fa-a6d4-3bd5598f7ae4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.380079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc8612-d236-41fa-a6d4-3bd5598f7ae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:24.381677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc8612-d236-41fa-a6d4-3bd5598f7ae4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:24.383254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 121/245] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:24.388909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc8612-d236-41fa-a6d4-3bd5598f7ae4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:24.400507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:24.426993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 125/246] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:24.427920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8b642-1de2-490c-afc0-a4cf72c966d6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:24.478678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:24.500046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc8612-d236-41fa-a6d4-3bd5598f7ae4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:24.534300 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a706c5-d205-43e5-9533-3ce30ba4d684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:24.534913 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a706c5-d205-43e5-9533-3ce30ba4d684 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.535234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a706c5-d205-43e5-9533-3ce30ba4d684 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.535512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a706c5-d205-43e5-9533-3ce30ba4d684 None admin] PATH_INFO: `/v3/groups` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.536143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a706c5-d205-43e5-9533-3ce30ba4d684 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:24.537641 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a706c5-d205-43e5-9533-3ce30ba4d684 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:24.540875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8b642-1de2-490c-afc0-a4cf72c966d6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:24.578224 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0f695-f76a-48be-9ba6-184d51e834d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:24.578859 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0f695-f76a-48be-9ba6-184d51e834d6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.579141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0f695-f76a-48be-9ba6-184d51e834d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.579413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0f695-f76a-48be-9ba6-184d51e834d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.589302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0f695-f76a-48be-9ba6-184d51e834d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cce0f695-f76a-48be-9ba6-184d51e834d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:24.594551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:24.595642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:24.596926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 126/247] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:24.601336 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0f695-f76a-48be-9ba6-184d51e834d6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:24.643491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:24.699384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0f695-f76a-48be-9ba6-184d51e834d6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:24.725932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 122/248] 10.222.0.30 () {60 vars in 1134 bytes} [Sat Oct 15 12:43:24 2022] POST /identity/v3/groups => generated 226 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:43:24.726893 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a706c5-d205-43e5-9533-3ce30ba4d684 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:24.733691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52765e-ffc8-4cda-8474-1538cf7c2e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:24.734324 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52765e-ffc8-4cda-8474-1538cf7c2e2c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.734598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52765e-ffc8-4cda-8474-1538cf7c2e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.734867 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52765e-ffc8-4cda-8474-1538cf7c2e2c None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.735547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52765e-ffc8-4cda-8474-1538cf7c2e2c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:24.735877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52765e-ffc8-4cda-8474-1538cf7c2e2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:24.737328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52765e-ffc8-4cda-8474-1538cf7c2e2c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:24.747378 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 127/249] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:24.753227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52765e-ffc8-4cda-8474-1538cf7c2e2c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:24.857536 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a706c5-d205-43e5-9533-3ce30ba4d684 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:24.877397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52765e-ffc8-4cda-8474-1538cf7c2e2c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:24.879132 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:24.895134 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911597c-cd47-4e82-abbb-a2439e1223b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:24.895843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911597c-cd47-4e82-abbb-a2439e1223b6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.896094 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911597c-cd47-4e82-abbb-a2439e1223b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.896323 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911597c-cd47-4e82-abbb-a2439e1223b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.913405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911597c-cd47-4e82-abbb-a2439e1223b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:24.917574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92424647-b9cb-4fbe-89e3-81553e8dc4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:24.918300 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92424647-b9cb-4fbe-89e3-81553e8dc4fb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:24.918648 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92424647-b9cb-4fbe-89e3-81553e8dc4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:24.918937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92424647-b9cb-4fbe-89e3-81553e8dc4fb None admin] PATH_INFO: `/v3/users/3d454bafc80f4af1a71aa4549424cac8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f911597c-cd47-4e82-abbb-a2439e1223b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:24.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:24.921035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:24.922762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:24.923369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92424647-b9cb-4fbe-89e3-81553e8dc4fb None admin] RBAC: Authorizing `identity:get_user(user_id=3d454bafc80f4af1a71aa4549424cac8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:24.923438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 123/250] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:24.923502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92424647-b9cb-4fbe-89e3-81553e8dc4fb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:24.923998 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 128/251] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:24 2022] GET /identity/v3/users/3d454bafc80f4af1a71aa4549424cac8 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:24.929089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911597c-cd47-4e82-abbb-a2439e1223b6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:24.931069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92424647-b9cb-4fbe-89e3-81553e8dc4fb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:24.932458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:25.062184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911597c-cd47-4e82-abbb-a2439e1223b6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:25.073662 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92424647-b9cb-4fbe-89e3-81553e8dc4fb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:25.103553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ffad6-5b4d-4e3b-8129-307f6eb8e818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:25.103553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ffad6-5b4d-4e3b-8129-307f6eb8e818 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:25.103908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ffad6-5b4d-4e3b-8129-307f6eb8e818 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:25.104102 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ffad6-5b4d-4e3b-8129-307f6eb8e818 None admin] PATH_INFO: `/v3/system/users/3d454bafc80f4af1a71aa4549424cac8/roles/8b0de894383f4c18a3363436c36841fa` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:25.106401 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ffad6-5b4d-4e3b-8129-307f6eb8e818 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3d454bafc80f4af1a71aa4549424cac8, role_id=8b0de894383f4c18a3363436c36841fa)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:25.108086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ffad6-5b4d-4e3b-8129-307f6eb8e818 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:25.124651 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55f102-1bf4-4b83-a29d-dc03fcb67e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:25.125662 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55f102-1bf4-4b83-a29d-dc03fcb67e2b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:25.126153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55f102-1bf4-4b83-a29d-dc03fcb67e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:25.126609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55f102-1bf4-4b83-a29d-dc03fcb67e2b None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:25.127747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55f102-1bf4-4b83-a29d-dc03fcb67e2b None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:25.128955 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55f102-1bf4-4b83-a29d-dc03fcb67e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:25.131036 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55f102-1bf4-4b83-a29d-dc03fcb67e2b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:25.142839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 129/252] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:25.148786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55f102-1bf4-4b83-a29d-dc03fcb67e2b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:25.183536 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:25.196505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 124/253] 10.222.0.30 () {58 vars in 1334 bytes} [Sat Oct 15 12:43:24 2022] PUT /identity/v3/system/users/3d454bafc80f4af1a71aa4549424cac8/roles/8b0de894383f4c18a3363436c36841fa => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 12:43:25.273956 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55f102-1bf4-4b83-a29d-dc03fcb67e2b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:25.314469 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31fffa-8b7a-47ee-b469-382f3ac91218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:25.315110 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31fffa-8b7a-47ee-b469-382f3ac91218 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:25.315394 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31fffa-8b7a-47ee-b469-382f3ac91218 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:25.315670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31fffa-8b7a-47ee-b469-382f3ac91218 None admin] PATH_INFO: `/v3/users/841dc4d48b7448b5801b69f695649a84` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:25.316843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31fffa-8b7a-47ee-b469-382f3ac91218 None admin] RBAC: Authorizing `identity:get_user(user_id=841dc4d48b7448b5801b69f695649a84)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:25.318354 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31fffa-8b7a-47ee-b469-382f3ac91218 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:25.320377 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 130/254] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:25 2022] GET /identity/v3/users/841dc4d48b7448b5801b69f695649a84 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:25.326260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ffad6-5b4d-4e3b-8129-307f6eb8e818 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:25.405278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:25.450851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ffad6-5b4d-4e3b-8129-307f6eb8e818 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:25.495885 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc95fa-8a47-4e4e-9b97-23c88890fa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:25.497044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc95fa-8a47-4e4e-9b97-23c88890fa70 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:25.497555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc95fa-8a47-4e4e-9b97-23c88890fa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:25.498109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc95fa-8a47-4e4e-9b97-23c88890fa70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:25.499194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc95fa-8a47-4e4e-9b97-23c88890fa70 None admin] RBAC: Adding query filter params (role.id=f1fda4fad003483dbc5843e16b6dc9d2, user.id=841dc4d48b7448b5801b69f695649a84, scope.system=all) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:25.499704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc95fa-8a47-4e4e-9b97-23c88890fa70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:25.501402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc95fa-8a47-4e4e-9b97-23c88890fa70 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:25.515441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 125/255] 10.222.0.30 () {56 vars in 1308 bytes} [Sat Oct 15 12:43:25 2022] GET /identity/v3/role_assignments?user.id=841dc4d48b7448b5801b69f695649a84&scope.system=all&role.id=f1fda4fad003483dbc5843e16b6dc9d2 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:27.391492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea955-7bee-42a7-85e7-41100bc64185 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:27.392020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea955-7bee-42a7-85e7-41100bc64185 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:27.392020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea955-7bee-42a7-85e7-41100bc64185 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:27.393655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 131/256] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:27.400086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9567e-56c0-4e1b-a8ee-d44f3130918e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:27.400911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9567e-56c0-4e1b-a8ee-d44f3130918e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:27.401405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9567e-56c0-4e1b-a8ee-d44f3130918e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:27.426490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bdb9567e-56c0-4e1b-a8ee-d44f3130918e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:27.428015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bdb9567e-56c0-4e1b-a8ee-d44f3130918e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:27.453441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c0b76-19d3-4d7d-9e54-f6cb30240095 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:27.453441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c0b76-19d3-4d7d-9e54-f6cb30240095 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:27.453709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c0b76-19d3-4d7d-9e54-f6cb30240095 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:27.455308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 132/257] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:27.460505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d0db7-09ae-42fe-be2b-95a7550585e9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:27.460787 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d0db7-09ae-42fe-be2b-95a7550585e9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:27.461066 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d0db7-09ae-42fe-be2b-95a7550585e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:27.487709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ee0d0db7-09ae-42fe-be2b-95a7550585e9 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:27.488940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ee0d0db7-09ae-42fe-be2b-95a7550585e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:27.500491 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 126/258] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 12:43:27.565531 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 133/259] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 15 12:43:27.829172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9567e-56c0-4e1b-a8ee-d44f3130918e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:27.838465 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9567e-56c0-4e1b-a8ee-d44f3130918e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:27.849435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e403ea7-0e8a-4440-ad96-e770d07880ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:27.850089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e403ea7-0e8a-4440-ad96-e770d07880ed admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:27.850370 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e403ea7-0e8a-4440-ad96-e770d07880ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:27.850648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e403ea7-0e8a-4440-ad96-e770d07880ed admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:27.866130 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e403ea7-0e8a-4440-ad96-e770d07880ed admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5e403ea7-0e8a-4440-ad96-e770d07880ed admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:27.871671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:27.872544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:27.873385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 127/260] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 12:43:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:27.880350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d0db7-09ae-42fe-be2b-95a7550585e9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:27.890173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d0db7-09ae-42fe-be2b-95a7550585e9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:27.901134 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:27.901774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:27.902054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:27.902865 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:27.903716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:27.904023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:27.905881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 12:43:27.905881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Oct 15 12:43:27.905881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: Oct 15 12:43:27.906815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:27.925086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 134/261] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 12:43:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:27.931473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e403ea7-0e8a-4440-ad96-e770d07880ed admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:27.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e403ea7-0e8a-4440-ad96-e770d07880ed admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:27.950786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889ef5a-4a27-4236-bb21-639419c66f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:27.951414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889ef5a-4a27-4236-bb21-639419c66f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:27.951696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889ef5a-4a27-4236-bb21-639419c66f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:27.951965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889ef5a-4a27-4236-bb21-639419c66f14 admin admin] PATH_INFO: `/v3/groups/d1a083a661a64292a236919e376ef23b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:27.961597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889ef5a-4a27-4236-bb21-639419c66f14 admin admin] RBAC: Authorizing `identity:get_group(group_id=d1a083a661a64292a236919e376ef23b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:27.963374 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-6889ef5a-4a27-4236-bb21-639419c66f14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 12:43:27.963374 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 15 12:43:27.963374 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: Oct 15 12:43:27.964603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889ef5a-4a27-4236-bb21-639419c66f14 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:27.967197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 128/262] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 12:43:27 2022] GET /identity/v3/groups/d1a083a661a64292a236919e376ef23b => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:27.972504 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:27.982674 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cc7e3-627e-4dc9-a705-6629e04a2770 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:27.987086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d280f-d36a-419a-a533-448ae49918d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:27.987719 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d280f-d36a-419a-a533-448ae49918d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:27.987994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d280f-d36a-419a-a533-448ae49918d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:27.988266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d280f-d36a-419a-a533-448ae49918d5 admin admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:27.989408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d280f-d36a-419a-a533-448ae49918d5 admin admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:27.991487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d280f-d36a-419a-a533-448ae49918d5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:27.993265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 135/263] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 12:43:27 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:27.999767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889ef5a-4a27-4236-bb21-639419c66f14 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.011140 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceb612-2c5e-4931-bc5d-dac7a5714b90 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.011419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceb612-2c5e-4931-bc5d-dac7a5714b90 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.011753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceb612-2c5e-4931-bc5d-dac7a5714b90 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.013292 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 136/264] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:43:28.016733 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889ef5a-4a27-4236-bb21-639419c66f14 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:28.021109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b84dc7-9ec3-4afe-8d0e-59a3da0acf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:28.021833 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84dc7-9ec3-4afe-8d0e-59a3da0acf3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.022111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84dc7-9ec3-4afe-8d0e-59a3da0acf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.022384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84dc7-9ec3-4afe-8d0e-59a3da0acf3b admin admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/groups/d1a083a661a64292a236919e376ef23b/roles/ffd57ebcb27d45ef9b9d642f409cc6d7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.029910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b84dc7-9ec3-4afe-8d0e-59a3da0acf3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c030436e634b6bb7d8548e2f2e6e41, group_id=d1a083a661a64292a236919e376ef23b, role_id=ffd57ebcb27d45ef9b9d642f409cc6d7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:28.030941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-f5b84dc7-9ec3-4afe-8d0e-59a3da0acf3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 12:43:28.030941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 15 12:43:28.030941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: Oct 15 12:43:28.032930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b84dc7-9ec3-4afe-8d0e-59a3da0acf3b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:28.041583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5e9d-4aa7-4553-b7ec-8c4f56a902b3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.041865 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5e9d-4aa7-4553-b7ec-8c4f56a902b3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.042126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5e9d-4aa7-4553-b7ec-8c4f56a902b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.065695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6c3a5e9d-4aa7-4553-b7ec-8c4f56a902b3 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:28.066901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c3a5e9d-4aa7-4553-b7ec-8c4f56a902b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:28.100653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:28.144651 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 129/265] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 12:43:27 2022] PUT /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/groups/d1a083a661a64292a236919e376ef23b/roles/ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:28.145431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b84dc7-9ec3-4afe-8d0e-59a3da0acf3b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.176106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 137/266] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:28.236299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:28.259245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b84dc7-9ec3-4afe-8d0e-59a3da0acf3b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:28.264848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca810eb-a9df-4172-a601-2915ff08712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:28.265540 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca810eb-a9df-4172-a601-2915ff08712e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.265818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca810eb-a9df-4172-a601-2915ff08712e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.266091 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca810eb-a9df-4172-a601-2915ff08712e admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.277027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeebbd6-5616-4152-a47c-2642eeebc316 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.281423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeebbd6-5616-4152-a47c-2642eeebc316 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.281423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeebbd6-5616-4152-a47c-2642eeebc316 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.281423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 138/267] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:43:28.285362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65003219-6734-484d-b86c-676e7844c0c6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.285715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65003219-6734-484d-b86c-676e7844c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.286065 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65003219-6734-484d-b86c-676e7844c0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.291344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca810eb-a9df-4172-a601-2915ff08712e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4ca810eb-a9df-4172-a601-2915ff08712e admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:28.297442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:28.302380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:28.303172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 130/268] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 12:43:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:28.308325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca810eb-a9df-4172-a601-2915ff08712e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.324799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca810eb-a9df-4172-a601-2915ff08712e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:28.329553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:28.330212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.330552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.330878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.331577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:28.331883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:28.332973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 12:43:28.332973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 15 12:43:28.332973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: Oct 15 12:43:28.334431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-65003219-6734-484d-b86c-676e7844c0c6 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:28.335582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-65003219-6734-484d-b86c-676e7844c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:28.338362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:28.356033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 131/269] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:43:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:28.358390 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.358795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 139/270] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 12:43:28.371580 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2edf3-e470-4525-8b32-93c2df547136 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:28.376320 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f730e-9633-4f5f-9af2-1c9afba97557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:28.377029 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f730e-9633-4f5f-9af2-1c9afba97557 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.377341 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f730e-9633-4f5f-9af2-1c9afba97557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.377613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f730e-9633-4f5f-9af2-1c9afba97557 admin admin] PATH_INFO: `/v3/groups/d1a083a661a64292a236919e376ef23b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.379093 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f730e-9633-4f5f-9af2-1c9afba97557 admin admin] RBAC: Authorizing `identity:get_group(group_id=d1a083a661a64292a236919e376ef23b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:28.385395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f730e-9633-4f5f-9af2-1c9afba97557 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:28.387957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 132/271] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 12:43:28 2022] GET /identity/v3/groups/d1a083a661a64292a236919e376ef23b => generated 226 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:28.394560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65003219-6734-484d-b86c-676e7844c0c6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.413954 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65003219-6734-484d-b86c-676e7844c0c6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:28.430818 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537e56f-027d-49a4-9ae1-07ddeba4e9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:28.431214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537e56f-027d-49a4-9ae1-07ddeba4e9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.431493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537e56f-027d-49a4-9ae1-07ddeba4e9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.431752 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537e56f-027d-49a4-9ae1-07ddeba4e9c8 admin admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.438244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537e56f-027d-49a4-9ae1-07ddeba4e9c8 admin admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:28.440390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537e56f-027d-49a4-9ae1-07ddeba4e9c8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:28.442480 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 140/272] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 12:43:28 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:28.448483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f730e-9633-4f5f-9af2-1c9afba97557 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.465685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f730e-9633-4f5f-9af2-1c9afba97557 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:28.470350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7971-67b7-4f50-ab6c-576ea92ee72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:28.471054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7971-67b7-4f50-ab6c-576ea92ee72b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.471330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7971-67b7-4f50-ab6c-576ea92ee72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.471649 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7971-67b7-4f50-ab6c-576ea92ee72b admin admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/groups/d1a083a661a64292a236919e376ef23b/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.473730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb7971-67b7-4f50-ab6c-576ea92ee72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c030436e634b6bb7d8548e2f2e6e41, group_id=d1a083a661a64292a236919e376ef23b, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:28.476547 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb7971-67b7-4f50-ab6c-576ea92ee72b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:28.578055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 133/273] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 12:43:28 2022] PUT /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41/groups/d1a083a661a64292a236919e376ef23b/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:28.724684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537e56f-027d-49a4-9ae1-07ddeba4e9c8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.793879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7971-67b7-4f50-ab6c-576ea92ee72b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.857978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537e56f-027d-49a4-9ae1-07ddeba4e9c8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:28.908745 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98f7d2-f7ac-4cc1-989a-91d218c9acf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:28.909670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98f7d2-f7ac-4cc1-989a-91d218c9acf9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.909946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98f7d2-f7ac-4cc1-989a-91d218c9acf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.910214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98f7d2-f7ac-4cc1-989a-91d218c9acf9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.918217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7971-67b7-4f50-ab6c-576ea92ee72b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:28.929674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bcabf-a46f-482e-98cd-cde16c8a8770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:28.930315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bcabf-a46f-482e-98cd-cde16c8a8770 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.930592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bcabf-a46f-482e-98cd-cde16c8a8770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:28.930862 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bcabf-a46f-482e-98cd-cde16c8a8770 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:28.934155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98f7d2-f7ac-4cc1-989a-91d218c9acf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ac98f7d2-f7ac-4cc1-989a-91d218c9acf9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:28.940318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:28.941214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:28.942020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 141/274] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:28.950252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98f7d2-f7ac-4cc1-989a-91d218c9acf9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.957232 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bcabf-a46f-482e-98cd-cde16c8a8770 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b39bcabf-a46f-482e-98cd-cde16c8a8770 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:28.962447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:28.963359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:28.964159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:28.968735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 134/275] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 12:43:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:28.974278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bcabf-a46f-482e-98cd-cde16c8a8770 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:28.983683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:28.990036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bcabf-a46f-482e-98cd-cde16c8a8770 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:28.998136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32db03-7800-45a8-94d0-cf810717a7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:28.999026 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32db03-7800-45a8-94d0-cf810717a7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:28.999644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32db03-7800-45a8-94d0-cf810717a7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:29.000146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32db03-7800-45a8-94d0-cf810717a7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:29.001020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32db03-7800-45a8-94d0-cf810717a7f4 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:29.001556 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32db03-7800-45a8-94d0-cf810717a7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:29.003512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32db03-7800-45a8-94d0-cf810717a7f4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:29.014743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 135/276] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:43:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:29.020281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32db03-7800-45a8-94d0-cf810717a7f4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:29.035857 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32db03-7800-45a8-94d0-cf810717a7f4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:29.055926 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ec863-f1f4-464f-b266-40a37e51ca07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:29.057156 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ec863-f1f4-464f-b266-40a37e51ca07 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:29.057481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ec863-f1f4-464f-b266-40a37e51ca07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:29.058876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ec863-f1f4-464f-b266-40a37e51ca07 admin admin] PATH_INFO: `/v3/groups/d1a083a661a64292a236919e376ef23b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:29.060658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ec863-f1f4-464f-b266-40a37e51ca07 admin admin] RBAC: Authorizing `identity:get_group(group_id=d1a083a661a64292a236919e376ef23b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:29.065347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ec863-f1f4-464f-b266-40a37e51ca07 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:29.070228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 136/277] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 12:43:29 2022] GET /identity/v3/groups/d1a083a661a64292a236919e376ef23b => generated 226 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:29.075851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ec863-f1f4-464f-b266-40a37e51ca07 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:29.089008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ec863-f1f4-464f-b266-40a37e51ca07 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:29.101793 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30da0c3-76b9-4f78-8058-a523263ba4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:29.102433 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30da0c3-76b9-4f78-8058-a523263ba4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:29.102952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30da0c3-76b9-4f78-8058-a523263ba4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:29.103228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30da0c3-76b9-4f78-8058-a523263ba4e3 admin admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:29.107250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30da0c3-76b9-4f78-8058-a523263ba4e3 admin admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:29.115115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30da0c3-76b9-4f78-8058-a523263ba4e3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:29.116878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 137/278] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 12:43:29 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:29.122698 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30da0c3-76b9-4f78-8058-a523263ba4e3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:29.143206 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30da0c3-76b9-4f78-8058-a523263ba4e3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:29.144865 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98f7d2-f7ac-4cc1-989a-91d218c9acf9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:29.162480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc0a1a-1da2-40a7-a95e-57ff6cd57934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:29.163176 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc0a1a-1da2-40a7-a95e-57ff6cd57934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:29.163443 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc0a1a-1da2-40a7-a95e-57ff6cd57934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:29.163718 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc0a1a-1da2-40a7-a95e-57ff6cd57934 admin admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147/groups/d1a083a661a64292a236919e376ef23b/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:29.170612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc0a1a-1da2-40a7-a95e-57ff6cd57934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5580ff9cebc43c6933eb6147b391147, group_id=d1a083a661a64292a236919e376ef23b, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:29.173337 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc0a1a-1da2-40a7-a95e-57ff6cd57934 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:29.199847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277bf05-d0e2-42f9-9b7b-7dc0e4d85fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:29.200817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277bf05-d0e2-42f9-9b7b-7dc0e4d85fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:29.201315 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277bf05-d0e2-42f9-9b7b-7dc0e4d85fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:29.201774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277bf05-d0e2-42f9-9b7b-7dc0e4d85fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:29.202605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277bf05-d0e2-42f9-9b7b-7dc0e4d85fc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:29.203119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277bf05-d0e2-42f9-9b7b-7dc0e4d85fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:29.204720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277bf05-d0e2-42f9-9b7b-7dc0e4d85fc5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:29.217443 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 142/279] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:29.223192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277bf05-d0e2-42f9-9b7b-7dc0e4d85fc5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:29.293510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 138/280] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 12:43:29 2022] PUT /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147/groups/d1a083a661a64292a236919e376ef23b/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:29.322704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:29.353889 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277bf05-d0e2-42f9-9b7b-7dc0e4d85fc5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:29.407407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50517398-fa35-4b08-b01d-2b3fef20ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:29.408056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50517398-fa35-4b08-b01d-2b3fef20ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:29.408316 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50517398-fa35-4b08-b01d-2b3fef20ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:29.408596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50517398-fa35-4b08-b01d-2b3fef20ddf4 None admin] PATH_INFO: `/v3/users/3d454bafc80f4af1a71aa4549424cac8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:29.410019 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50517398-fa35-4b08-b01d-2b3fef20ddf4 None admin] RBAC: Authorizing `identity:get_user(user_id=3d454bafc80f4af1a71aa4549424cac8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:29.411546 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50517398-fa35-4b08-b01d-2b3fef20ddf4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:29.413128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 143/281] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:43:29 2022] GET /identity/v3/users/3d454bafc80f4af1a71aa4549424cac8 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:29.419512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc0a1a-1da2-40a7-a95e-57ff6cd57934 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:29.536543 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:29.590395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc0a1a-1da2-40a7-a95e-57ff6cd57934 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:29.641334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25191ed9-0821-46ef-b2ab-5514e7f9be04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:29.641982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25191ed9-0821-46ef-b2ab-5514e7f9be04 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:29.642261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25191ed9-0821-46ef-b2ab-5514e7f9be04 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:29.642534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25191ed9-0821-46ef-b2ab-5514e7f9be04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:29.643294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25191ed9-0821-46ef-b2ab-5514e7f9be04 None admin] RBAC: Adding query filter params (role.id=8b0de894383f4c18a3363436c36841fa, user.id=3d454bafc80f4af1a71aa4549424cac8, scope.system=all) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:29.643607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25191ed9-0821-46ef-b2ab-5514e7f9be04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:29.645151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25191ed9-0821-46ef-b2ab-5514e7f9be04 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:29.664863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 139/282] 10.222.0.30 () {56 vars in 1308 bytes} [Sat Oct 15 12:43:29 2022] GET /identity/v3/role_assignments?user.id=3d454bafc80f4af1a71aa4549424cac8&scope.system=all&role.id=8b0de894383f4c18a3363436c36841fa => generated 510 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:30.060014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea82e4-9d54-4c74-852e-f4d4a9a9fbd6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.060430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea82e4-9d54-4c74-852e-f4d4a9a9fbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.060623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea82e4-9d54-4c74-852e-f4d4a9a9fbd6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.062389 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 144/283] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:30.070959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0689b-6aa4-4077-bba9-dc1d1b0935dd None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.071262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0689b-6aa4-4077-bba9-dc1d1b0935dd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.071507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0689b-6aa4-4077-bba9-dc1d1b0935dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.094265 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9ef0689b-6aa4-4077-bba9-dc1d1b0935dd None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:30.095580 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9ef0689b-6aa4-4077-bba9-dc1d1b0935dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:30.134159 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4489b3-70a0-4164-80a5-88576b9322d5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.134460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4489b3-70a0-4164-80a5-88576b9322d5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.134708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4489b3-70a0-4164-80a5-88576b9322d5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.136116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 145/284] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:43:30.141696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ee34c-b54b-49ba-af83-29dc504445f5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.141978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ee34c-b54b-49ba-af83-29dc504445f5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.142246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ee34c-b54b-49ba-af83-29dc504445f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.167772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6d4ee34c-b54b-49ba-af83-29dc504445f5 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:30.171277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6d4ee34c-b54b-49ba-af83-29dc504445f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:30.189002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 140/285] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 12:43:30.242189 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 146/286] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 15 12:43:30.532103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0689b-6aa4-4077-bba9-dc1d1b0935dd None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.548392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0689b-6aa4-4077-bba9-dc1d1b0935dd None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.559604 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe353f81-279d-4c4d-aef1-ab320448bf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.560494 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353f81-279d-4c4d-aef1-ab320448bf16 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.560991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353f81-279d-4c4d-aef1-ab320448bf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.561494 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353f81-279d-4c4d-aef1-ab320448bf16 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.573975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe353f81-279d-4c4d-aef1-ab320448bf16 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-fe353f81-279d-4c4d-aef1-ab320448bf16 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:30.579606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:30.580471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:30.581299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:30.583829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 141/287] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 12:43:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:30.589771 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ee34c-b54b-49ba-af83-29dc504445f5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.599855 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ee34c-b54b-49ba-af83-29dc504445f5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.604729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7867c9-c1ce-40a1-81b5-55ef98c96cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.607297 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7867c9-c1ce-40a1-81b5-55ef98c96cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.607798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7867c9-c1ce-40a1-81b5-55ef98c96cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.608272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7867c9-c1ce-40a1-81b5-55ef98c96cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.609163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7867c9-c1ce-40a1-81b5-55ef98c96cf1 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:30.609695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7867c9-c1ce-40a1-81b5-55ef98c96cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:30.611706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7867c9-c1ce-40a1-81b5-55ef98c96cf1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:30.621710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 147/288] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 12:43:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:30.627436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe353f81-279d-4c4d-aef1-ab320448bf16 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.639428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe353f81-279d-4c4d-aef1-ab320448bf16 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.648003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336b444-9f49-46de-9986-2ad118f4010e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.648915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336b444-9f49-46de-9986-2ad118f4010e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.649429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336b444-9f49-46de-9986-2ad118f4010e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.649903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336b444-9f49-46de-9986-2ad118f4010e admin admin] PATH_INFO: `/v3/groups/d1a083a661a64292a236919e376ef23b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.651250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336b444-9f49-46de-9986-2ad118f4010e admin admin] RBAC: Authorizing `identity:get_group(group_id=d1a083a661a64292a236919e376ef23b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:30.653523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336b444-9f49-46de-9986-2ad118f4010e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:30.655322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 142/289] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 12:43:30 2022] GET /identity/v3/groups/d1a083a661a64292a236919e376ef23b => generated 226 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:30.661310 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7867c9-c1ce-40a1-81b5-55ef98c96cf1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.672313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7867c9-c1ce-40a1-81b5-55ef98c96cf1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.678196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2d134-4823-4dd6-9d6c-82d6579fcd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.679073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2d134-4823-4dd6-9d6c-82d6579fcd25 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.679558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2d134-4823-4dd6-9d6c-82d6579fcd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.680033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2d134-4823-4dd6-9d6c-82d6579fcd25 admin admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.681415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2d134-4823-4dd6-9d6c-82d6579fcd25 admin admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:30.682924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336b444-9f49-46de-9986-2ad118f4010e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.691329 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2d134-4823-4dd6-9d6c-82d6579fcd25 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:30.697213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 148/290] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 12:43:30 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:30.698116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336b444-9f49-46de-9986-2ad118f4010e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.703851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe8c39-cf94-420b-bdf8-13803d12ed92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.704823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe8c39-cf94-420b-bdf8-13803d12ed92 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.705339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe8c39-cf94-420b-bdf8-13803d12ed92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.705814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe8c39-cf94-420b-bdf8-13803d12ed92 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.715952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2d134-4823-4dd6-9d6c-82d6579fcd25 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.721751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe8c39-cf94-420b-bdf8-13803d12ed92 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-31fe8c39-cf94-420b-bdf8-13803d12ed92 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:30.727622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:30.728506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:30.729360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:30.732512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 143/291] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 12:43:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:30.737337 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2d134-4823-4dd6-9d6c-82d6579fcd25 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.738587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe8c39-cf94-420b-bdf8-13803d12ed92 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.746523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581487ae-d6c6-4efb-bef5-33fabc2c3b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.747268 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581487ae-d6c6-4efb-bef5-33fabc2c3b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.747607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581487ae-d6c6-4efb-bef5-33fabc2c3b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.747977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581487ae-d6c6-4efb-bef5-33fabc2c3b92 admin admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147/groups/d1a083a661a64292a236919e376ef23b/roles/ffd57ebcb27d45ef9b9d642f409cc6d7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.751396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581487ae-d6c6-4efb-bef5-33fabc2c3b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5580ff9cebc43c6933eb6147b391147, group_id=d1a083a661a64292a236919e376ef23b, role_id=ffd57ebcb27d45ef9b9d642f409cc6d7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:30.752843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-581487ae-d6c6-4efb-bef5-33fabc2c3b92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 12:43:30.752843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Oct 15 12:43:30.752843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: Oct 15 12:43:30.755956 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581487ae-d6c6-4efb-bef5-33fabc2c3b92 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:30.762413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe8c39-cf94-420b-bdf8-13803d12ed92 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.768531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe7ccb-60db-475c-a528-40dc920dbf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.769146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe7ccb-60db-475c-a528-40dc920dbf94 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.769510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe7ccb-60db-475c-a528-40dc920dbf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.769783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe7ccb-60db-475c-a528-40dc920dbf94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.770452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe7ccb-60db-475c-a528-40dc920dbf94 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:30.770811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe7ccb-60db-475c-a528-40dc920dbf94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:30.772610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe7ccb-60db-475c-a528-40dc920dbf94 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:30.782529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 144/292] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 12:43:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:30.788592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe7ccb-60db-475c-a528-40dc920dbf94 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.799524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe7ccb-60db-475c-a528-40dc920dbf94 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.806317 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bd036-06d1-41a8-aee9-8f7b4debeb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.806948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bd036-06d1-41a8-aee9-8f7b4debeb9c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.807275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bd036-06d1-41a8-aee9-8f7b4debeb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.807611 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bd036-06d1-41a8-aee9-8f7b4debeb9c admin admin] PATH_INFO: `/v3/groups/46f7c5409b014784a6d5f181d49435ea` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.813490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bd036-06d1-41a8-aee9-8f7b4debeb9c admin admin] RBAC: Authorizing `identity:get_group(group_id=46f7c5409b014784a6d5f181d49435ea)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:30.815511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bd036-06d1-41a8-aee9-8f7b4debeb9c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:30.817491 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 145/293] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 12:43:30 2022] GET /identity/v3/groups/46f7c5409b014784a6d5f181d49435ea => generated 229 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:30.822539 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bd036-06d1-41a8-aee9-8f7b4debeb9c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.841496 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 149/294] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 12:43:30 2022] PUT /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147/groups/d1a083a661a64292a236919e376ef23b/roles/ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:30.842027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bd036-06d1-41a8-aee9-8f7b4debeb9c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.870099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:30.920202 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36d615-cda2-4800-89ec-f464fcb56c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.920894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36d615-cda2-4800-89ec-f464fcb56c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.921172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36d615-cda2-4800-89ec-f464fcb56c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.921462 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36d615-cda2-4800-89ec-f464fcb56c86 admin admin] PATH_INFO: `/v3/projects/90479a5430a046cea76f80fced5060d9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.922308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36d615-cda2-4800-89ec-f464fcb56c86 admin admin] RBAC: Authorizing `identity:get_project(project_id=90479a5430a046cea76f80fced5060d9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:30.924221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36d615-cda2-4800-89ec-f464fcb56c86 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:30.926027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 146/295] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 12:43:30 2022] GET /identity/v3/projects/90479a5430a046cea76f80fced5060d9 => generated 344 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:30.932023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581487ae-d6c6-4efb-bef5-33fabc2c3b92 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:30.945941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581487ae-d6c6-4efb-bef5-33fabc2c3b92 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:30.951844 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff0ce0-f88c-45ec-96f6-813b844f31ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:30.952666 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff0ce0-f88c-45ec-96f6-813b844f31ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:30.952954 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff0ce0-f88c-45ec-96f6-813b844f31ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:30.953289 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff0ce0-f88c-45ec-96f6-813b844f31ac admin admin] PATH_INFO: `/v3/projects/90479a5430a046cea76f80fced5060d9/groups/46f7c5409b014784a6d5f181d49435ea/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:30.955344 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff0ce0-f88c-45ec-96f6-813b844f31ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=90479a5430a046cea76f80fced5060d9, group_id=46f7c5409b014784a6d5f181d49435ea, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:30.958192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff0ce0-f88c-45ec-96f6-813b844f31ac admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:31.036091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 150/296] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 12:43:30 2022] PUT /identity/v3/projects/90479a5430a046cea76f80fced5060d9/groups/46f7c5409b014784a6d5f181d49435ea/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 12:43:31.572196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451cec5-1212-413a-9149-22cba74ce08f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:31.572721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451cec5-1212-413a-9149-22cba74ce08f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:31.572721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451cec5-1212-413a-9149-22cba74ce08f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:31.574518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 147/297] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:31.580396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee8306-9c40-4fec-944c-0c3f46d84953 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:31.580689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee8306-9c40-4fec-944c-0c3f46d84953 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:31.580965 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee8306-9c40-4fec-944c-0c3f46d84953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:31.601609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-33ee8306-9c40-4fec-944c-0c3f46d84953 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:31.602800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-33ee8306-9c40-4fec-944c-0c3f46d84953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:31.611715 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91aadd-ae1c-4a38-a1f5-e2f36509e1bd None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:31.612007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91aadd-ae1c-4a38-a1f5-e2f36509e1bd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:31.612255 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91aadd-ae1c-4a38-a1f5-e2f36509e1bd None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:31.614062 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 148/298] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:31.618759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798399a-96fb-400b-9db0-9197eb1da30b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:31.619040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798399a-96fb-400b-9db0-9197eb1da30b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:31.619310 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798399a-96fb-400b-9db0-9197eb1da30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:31.646254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3798399a-96fb-400b-9db0-9197eb1da30b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:31.648288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3798399a-96fb-400b-9db0-9197eb1da30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:31.705080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:31.753464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 151/299] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:31.771719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 149/300] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:32.048174 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee8306-9c40-4fec-944c-0c3f46d84953 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:32.172962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee8306-9c40-4fec-944c-0c3f46d84953 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:32.219323 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b3e59-33ad-48b9-898e-1ce7df7301d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:32.219969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b3e59-33ad-48b9-898e-1ce7df7301d0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:32.220247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b3e59-33ad-48b9-898e-1ce7df7301d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:32.220521 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b3e59-33ad-48b9-898e-1ce7df7301d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:32.233534 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b3e59-33ad-48b9-898e-1ce7df7301d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-fd2b3e59-33ad-48b9-898e-1ce7df7301d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:32.238366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:32.239229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:32.240027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:32.244312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 152/301] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:43:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:32.245752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798399a-96fb-400b-9db0-9197eb1da30b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:32.268906 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b3e59-33ad-48b9-898e-1ce7df7301d0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:32.324267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:32.391494 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:32.398034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798399a-96fb-400b-9db0-9197eb1da30b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:32.414068 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b3e59-33ad-48b9-898e-1ce7df7301d0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:32.454843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc015c83-640b-4e71-9349-39babd305fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:32.455500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc015c83-640b-4e71-9349-39babd305fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:32.455782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc015c83-640b-4e71-9349-39babd305fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:32.456056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc015c83-640b-4e71-9349-39babd305fb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:32.457838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b563cd4-6135-4439-b690-537931e4a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:32.458450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b563cd4-6135-4439-b690-537931e4a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:32.458726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b563cd4-6135-4439-b690-537931e4a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:32.458998 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b563cd4-6135-4439-b690-537931e4a2a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:32.459667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b563cd4-6135-4439-b690-537931e4a2a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:32.459972 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b563cd4-6135-4439-b690-537931e4a2a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:32.461382 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b563cd4-6135-4439-b690-537931e4a2a1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:32.472956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 150/302] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:32.474885 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547a92a-129e-4b9c-b2c6-2ccffb9067b1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:32.475180 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547a92a-129e-4b9c-b2c6-2ccffb9067b1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:32.475920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc015c83-640b-4e71-9349-39babd305fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:32.477284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547a92a-129e-4b9c-b2c6-2ccffb9067b1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:32.478653 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 151/303] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-dc015c83-640b-4e71-9349-39babd305fb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:32.481660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:32.482562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:32.483390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:32.485913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 153/304] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:32.486187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547a92a-129e-4b9c-b2c6-2ccffb9067b1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:32.487321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9099e-e5e9-4f8f-aee0-0406e6418545 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:32.487651 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9099e-e5e9-4f8f-aee0-0406e6418545 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:32.487932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9099e-e5e9-4f8f-aee0-0406e6418545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:32.507107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6ec9099e-e5e9-4f8f-aee0-0406e6418545 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:32.508327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6ec9099e-e5e9-4f8f-aee0-0406e6418545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:32.584627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 154/305] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:32.585477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9099e-e5e9-4f8f-aee0-0406e6418545 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:32.632557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:32.633244 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547a92a-129e-4b9c-b2c6-2ccffb9067b1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:32.674207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016c024-afa7-4780-8ff4-685faf31934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:32.675237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016c024-afa7-4780-8ff4-685faf31934a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:32.675420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016c024-afa7-4780-8ff4-685faf31934a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:32.675791 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016c024-afa7-4780-8ff4-685faf31934a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:32.707685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:32.744213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9099e-e5e9-4f8f-aee0-0406e6418545 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:32.751656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016c024-afa7-4780-8ff4-685faf31934a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:32.753416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016c024-afa7-4780-8ff4-685faf31934a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:32.803216 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69199b-cb12-40ec-b7a1-58075a28bd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:32.803905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69199b-cb12-40ec-b7a1-58075a28bd97 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:32.804173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69199b-cb12-40ec-b7a1-58075a28bd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:32.804453 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69199b-cb12-40ec-b7a1-58075a28bd97 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:32.805119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69199b-cb12-40ec-b7a1-58075a28bd97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:32.805452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69199b-cb12-40ec-b7a1-58075a28bd97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:32.806840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69199b-cb12-40ec-b7a1-58075a28bd97 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:32.820409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 155/306] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:32.826778 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69199b-cb12-40ec-b7a1-58075a28bd97 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:32.862384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 152/307] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:32.868330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016c024-afa7-4780-8ff4-685faf31934a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:32.948959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:32.962333 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:32.973239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69199b-cb12-40ec-b7a1-58075a28bd97 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:33.009399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016c024-afa7-4780-8ff4-685faf31934a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:33.013311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde180-ea2c-49df-8271-a315521a9672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:33.013465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde180-ea2c-49df-8271-a315521a9672 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.013636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde180-ea2c-49df-8271-a315521a9672 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.013905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde180-ea2c-49df-8271-a315521a9672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.053446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c3de3-f3ce-4e49-ab3b-b378892a5edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:33.054568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c3de3-f3ce-4e49-ab3b-b378892a5edf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.054568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c3de3-f3ce-4e49-ab3b-b378892a5edf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.054568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c3de3-f3ce-4e49-ab3b-b378892a5edf None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.056422 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c3de3-f3ce-4e49-ab3b-b378892a5edf None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:33.060262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c3de3-f3ce-4e49-ab3b-b378892a5edf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:33.065307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 153/308] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:32 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:33.067062 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c3de3-f3ce-4e49-ab3b-b378892a5edf None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:33.106750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde180-ea2c-49df-8271-a315521a9672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:33.109795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde180-ea2c-49df-8271-a315521a9672 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:33.188858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c3de3-f3ce-4e49-ab3b-b378892a5edf None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:33.218885 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:33.227451 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaca271-d9d2-430f-8062-c3237bd00098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:33.228395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaca271-d9d2-430f-8062-c3237bd00098 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.228840 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaca271-d9d2-430f-8062-c3237bd00098 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.229301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaca271-d9d2-430f-8062-c3237bd00098 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.234317 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:33.242450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaca271-d9d2-430f-8062-c3237bd00098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:33.247160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 156/309] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:33.248769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde180-ea2c-49df-8271-a315521a9672 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ccaca271-d9d2-430f-8062-c3237bd00098 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:33.249275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:33.250303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 12:43:33.251115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:33.253325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 154/310] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:33.254297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaca271-d9d2-430f-8062-c3237bd00098 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:33.364324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaca271-d9d2-430f-8062-c3237bd00098 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:33.366330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde180-ea2c-49df-8271-a315521a9672 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:33.407470 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ac291-b8e0-4519-ae09-9b91397970f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:33.407673 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf115207-3ad7-40de-b0f8-e903a97f411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:33.408272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf115207-3ad7-40de-b0f8-e903a97f411c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.408549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf115207-3ad7-40de-b0f8-e903a97f411c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.408820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf115207-3ad7-40de-b0f8-e903a97f411c None admin] PATH_INFO: `/v3/groups/d1a083a661a64292a236919e376ef23b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.409516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ac291-b8e0-4519-ae09-9b91397970f3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.409992 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ac291-b8e0-4519-ae09-9b91397970f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.410250 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf115207-3ad7-40de-b0f8-e903a97f411c None admin] RBAC: Authorizing `identity:get_group(group_id=d1a083a661a64292a236919e376ef23b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:33.410658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ac291-b8e0-4519-ae09-9b91397970f3 None admin] PATH_INFO: `/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.411737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf115207-3ad7-40de-b0f8-e903a97f411c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:33.412331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ac291-b8e0-4519-ae09-9b91397970f3 None admin] RBAC: Authorizing `identity:get_project(project_id=a1c030436e634b6bb7d8548e2f2e6e41)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:33.413463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 157/311] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 12:43:33 2022] GET /identity/v3/groups/d1a083a661a64292a236919e376ef23b => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:33.414260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf115207-3ad7-40de-b0f8-e903a97f411c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:33.414489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ac291-b8e0-4519-ae09-9b91397970f3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:33.416472 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 155/312] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:33 2022] GET /identity/v3/projects/a1c030436e634b6bb7d8548e2f2e6e41 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:33.422252 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ac291-b8e0-4519-ae09-9b91397970f3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:33.478847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:33.489592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:33.523162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf115207-3ad7-40de-b0f8-e903a97f411c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:33.535785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ac291-b8e0-4519-ae09-9b91397970f3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:33.571925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d75470-23e4-468b-b589-68b08bc1a7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:33.572656 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d75470-23e4-468b-b589-68b08bc1a7db None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.572945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d75470-23e4-468b-b589-68b08bc1a7db None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.573369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d75470-23e4-468b-b589-68b08bc1a7db None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.574592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d75470-23e4-468b-b589-68b08bc1a7db None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:33.574909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d75470-23e4-468b-b589-68b08bc1a7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:33.576296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d75470-23e4-468b-b589-68b08bc1a7db None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:33.579253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d6798-a431-408f-95d4-5cc794919261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:33.579877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d6798-a431-408f-95d4-5cc794919261 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.580157 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d6798-a431-408f-95d4-5cc794919261 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.580429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d6798-a431-408f-95d4-5cc794919261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.581157 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d6798-a431-408f-95d4-5cc794919261 None admin] RBAC: Adding query filter params (group.id=d1a083a661a64292a236919e376ef23b, role.id=ffd57ebcb27d45ef9b9d642f409cc6d7, scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:33.581498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d6798-a431-408f-95d4-5cc794919261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:33.583070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d6798-a431-408f-95d4-5cc794919261 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:33.586527 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 158/313] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:43:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:33.587850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d75470-23e4-468b-b589-68b08bc1a7db None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:33.590202 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 156/314] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 12:43:33 2022] GET /identity/v3/role_assignments?group.id=d1a083a661a64292a236919e376ef23b&scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41&role.id=ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:33.591552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86404d1a-27b8-4fbb-b667-808667cbad9e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.591842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86404d1a-27b8-4fbb-b667-808667cbad9e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.592121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86404d1a-27b8-4fbb-b667-808667cbad9e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.593693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 157/315] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:33.594439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86404d1a-27b8-4fbb-b667-808667cbad9e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:33.728218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86404d1a-27b8-4fbb-b667-808667cbad9e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:33.733030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d75470-23e4-468b-b589-68b08bc1a7db None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:33.752610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:33.757682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:33.771315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9010ac9-4bab-4608-b137-33702fd48947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:33.772043 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9010ac9-4bab-4608-b137-33702fd48947 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9010ac9-4bab-4608-b137-33702fd48947 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.772502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553efa95-d65c-480a-a890-7949f9b64e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:33.772623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9010ac9-4bab-4608-b137-33702fd48947 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.773091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553efa95-d65c-480a-a890-7949f9b64e14 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.773406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553efa95-d65c-480a-a890-7949f9b64e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.773685 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553efa95-d65c-480a-a890-7949f9b64e14 None admin] PATH_INFO: `/v3/groups/d1a083a661a64292a236919e376ef23b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.774825 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553efa95-d65c-480a-a890-7949f9b64e14 None admin] RBAC: Authorizing `identity:get_group(group_id=d1a083a661a64292a236919e376ef23b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:33.776363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553efa95-d65c-480a-a890-7949f9b64e14 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:33.778782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 159/316] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 12:43:33 2022] GET /identity/v3/groups/d1a083a661a64292a236919e376ef23b => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:33.780081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d9623-90cd-47ac-920b-75fdf34bb9fb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:33.780365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d9623-90cd-47ac-920b-75fdf34bb9fb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:33.780639 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d9623-90cd-47ac-920b-75fdf34bb9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:33.801626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-870d9623-90cd-47ac-920b-75fdf34bb9fb None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:33.802869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-870d9623-90cd-47ac-920b-75fdf34bb9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:33.851412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9010ac9-4bab-4608-b137-33702fd48947 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:33.852975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9010ac9-4bab-4608-b137-33702fd48947 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:33.877861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 160/317] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:33.878715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d9623-90cd-47ac-920b-75fdf34bb9fb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:33.983163 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 158/318] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:33.988208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9010ac9-4bab-4608-b137-33702fd48947 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:34.014195 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:34.022325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d9623-90cd-47ac-920b-75fdf34bb9fb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:34.048113 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:34.068330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbf8c2-def2-4216-bfa8-8b91fe349591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:34.069604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf8c2-def2-4216-bfa8-8b91fe349591 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.090415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf8c2-def2-4216-bfa8-8b91fe349591 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.090415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf8c2-def2-4216-bfa8-8b91fe349591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.090415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbf8c2-def2-4216-bfa8-8b91fe349591 None admin] RBAC: Adding query filter params (group.id=d1a083a661a64292a236919e376ef23b, role.id=f1fda4fad003483dbc5843e16b6dc9d2, scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:34.090415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbf8c2-def2-4216-bfa8-8b91fe349591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:34.090415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbf8c2-def2-4216-bfa8-8b91fe349591 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:34.090975 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 161/319] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 12:43:33 2022] GET /identity/v3/role_assignments?group.id=d1a083a661a64292a236919e376ef23b&scope.project.id=a1c030436e634b6bb7d8548e2f2e6e41&role.id=f1fda4fad003483dbc5843e16b6dc9d2 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:34.115211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9010ac9-4bab-4608-b137-33702fd48947 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:34.161035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee952eb4-3546-4770-8aee-249382abcf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:34.162059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee952eb4-3546-4770-8aee-249382abcf71 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.162529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee952eb4-3546-4770-8aee-249382abcf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.163003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee952eb4-3546-4770-8aee-249382abcf71 None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.164492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee952eb4-3546-4770-8aee-249382abcf71 None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:34.166212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee952eb4-3546-4770-8aee-249382abcf71 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:34.168585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 159/320] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:33 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:34.174474 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbf8c2-def2-4216-bfa8-8b91fe349591 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:34.207368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee952eb4-3546-4770-8aee-249382abcf71 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:34.320402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbf8c2-def2-4216-bfa8-8b91fe349591 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:34.325248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:34.337934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee952eb4-3546-4770-8aee-249382abcf71 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:34.360188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e65972-8bc6-44aa-8466-0a3b7d727c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:34.360843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e65972-8bc6-44aa-8466-0a3b7d727c83 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.361122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e65972-8bc6-44aa-8466-0a3b7d727c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.361447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e65972-8bc6-44aa-8466-0a3b7d727c83 None admin] PATH_INFO: `/v3/groups/d1a083a661a64292a236919e376ef23b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.362629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e65972-8bc6-44aa-8466-0a3b7d727c83 None admin] RBAC: Authorizing `identity:get_group(group_id=d1a083a661a64292a236919e376ef23b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:34.364121 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e65972-8bc6-44aa-8466-0a3b7d727c83 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:34.366118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 162/321] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 12:43:34 2022] GET /identity/v3/groups/d1a083a661a64292a236919e376ef23b => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:34.372658 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c376c-22b5-4992-bb60-77005360286b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.372944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c376c-22b5-4992-bb60-77005360286b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.373256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c376c-22b5-4992-bb60-77005360286b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.374868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 163/322] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:43:34.375601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c376c-22b5-4992-bb60-77005360286b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:34.377701 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c61b1-5569-4785-9b05-07026cc5d2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:34.378621 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c61b1-5569-4785-9b05-07026cc5d2eb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.379091 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c61b1-5569-4785-9b05-07026cc5d2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.379548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c61b1-5569-4785-9b05-07026cc5d2eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.394828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c61b1-5569-4785-9b05-07026cc5d2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-932c61b1-5569-4785-9b05-07026cc5d2eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:34.400400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:34.401574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:34.402476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:34.403502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 160/323] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:43:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:34.404692 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05875c39-cd8b-47ed-bb37-d6a0e09c65b4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.404982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05875c39-cd8b-47ed-bb37-d6a0e09c65b4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.405335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05875c39-cd8b-47ed-bb37-d6a0e09c65b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.427607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-05875c39-cd8b-47ed-bb37-d6a0e09c65b4 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:34.428818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-05875c39-cd8b-47ed-bb37-d6a0e09c65b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:34.442936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:34.502773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c376c-22b5-4992-bb60-77005360286b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:34.506699 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 161/324] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:34.507717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05875c39-cd8b-47ed-bb37-d6a0e09c65b4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:34.560758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad9b49-ba52-4ef8-b1de-ae0c223797d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:34.561422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad9b49-ba52-4ef8-b1de-ae0c223797d2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.561708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad9b49-ba52-4ef8-b1de-ae0c223797d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.562545 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad9b49-ba52-4ef8-b1de-ae0c223797d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.562710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad9b49-ba52-4ef8-b1de-ae0c223797d2 None admin] RBAC: Adding query filter params (group.id=d1a083a661a64292a236919e376ef23b, role.id=f1fda4fad003483dbc5843e16b6dc9d2, scope.project.id=e5580ff9cebc43c6933eb6147b391147) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:34.563023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad9b49-ba52-4ef8-b1de-ae0c223797d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:34.564467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad9b49-ba52-4ef8-b1de-ae0c223797d2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:34.575277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 164/325] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 12:43:34 2022] GET /identity/v3/role_assignments?group.id=d1a083a661a64292a236919e376ef23b&scope.project.id=e5580ff9cebc43c6933eb6147b391147&role.id=f1fda4fad003483dbc5843e16b6dc9d2 => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:34.614948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:34.640106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05875c39-cd8b-47ed-bb37-d6a0e09c65b4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:34.680972 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b6c11-2566-4245-ad37-9d784ff14624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:34.681727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b6c11-2566-4245-ad37-9d784ff14624 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.682013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b6c11-2566-4245-ad37-9d784ff14624 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.682288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b6c11-2566-4245-ad37-9d784ff14624 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.682971 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b6c11-2566-4245-ad37-9d784ff14624 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:34.683279 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b6c11-2566-4245-ad37-9d784ff14624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:34.685368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b6c11-2566-4245-ad37-9d784ff14624 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:34.694866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 162/326] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:34.701837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad9b49-ba52-4ef8-b1de-ae0c223797d2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:34.791244 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b6c11-2566-4245-ad37-9d784ff14624 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:34.828010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad9b49-ba52-4ef8-b1de-ae0c223797d2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:34.846858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:34.866624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fb63a-8aa8-4aff-8b51-62e9a51fd5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:34.867247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fb63a-8aa8-4aff-8b51-62e9a51fd5ef None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.867528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fb63a-8aa8-4aff-8b51-62e9a51fd5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.867806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fb63a-8aa8-4aff-8b51-62e9a51fd5ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.923057 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b6c11-2566-4245-ad37-9d784ff14624 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:34.949232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fb63a-8aa8-4aff-8b51-62e9a51fd5ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:34.950711 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fb63a-8aa8-4aff-8b51-62e9a51fd5ef None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:34.963064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca40cd4-1b0d-41f9-882f-f1698b49c71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:34.963266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca40cd4-1b0d-41f9-882f-f1698b49c71b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:34.963750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca40cd4-1b0d-41f9-882f-f1698b49c71b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:34.964034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca40cd4-1b0d-41f9-882f-f1698b49c71b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:34.974257 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:34.978225 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca40cd4-1b0d-41f9-882f-f1698b49c71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-eca40cd4-1b0d-41f9-882f-f1698b49c71b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:34.984196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:34.990605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:34.991429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 163/327] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:43:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:34.993183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca40cd4-1b0d-41f9-882f-f1698b49c71b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:35.071762 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 165/328] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:35.079500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fb63a-8aa8-4aff-8b51-62e9a51fd5ef None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:35.101462 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:35.113416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca40cd4-1b0d-41f9-882f-f1698b49c71b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:35.154494 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d59f9-c8ce-404e-b628-d7ef8fe34fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:35.155116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d59f9-c8ce-404e-b628-d7ef8fe34fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:35.155418 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d59f9-c8ce-404e-b628-d7ef8fe34fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:35.155678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d59f9-c8ce-404e-b628-d7ef8fe34fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:35.156351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d59f9-c8ce-404e-b628-d7ef8fe34fc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:35.156660 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d59f9-c8ce-404e-b628-d7ef8fe34fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:35.158079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d59f9-c8ce-404e-b628-d7ef8fe34fc9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:35.167337 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 164/329] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:35.173562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d59f9-c8ce-404e-b628-d7ef8fe34fc9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:35.203397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fb63a-8aa8-4aff-8b51-62e9a51fd5ef None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:35.244075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2add628-3b31-4ff2-a4ac-3cd5ddeb197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:35.244655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2add628-3b31-4ff2-a4ac-3cd5ddeb197b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:35.244938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2add628-3b31-4ff2-a4ac-3cd5ddeb197b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:35.245277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2add628-3b31-4ff2-a4ac-3cd5ddeb197b None admin] PATH_INFO: `/v3/projects/e5580ff9cebc43c6933eb6147b391147` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:35.245661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:35.246532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2add628-3b31-4ff2-a4ac-3cd5ddeb197b None admin] RBAC: Authorizing `identity:get_project(project_id=e5580ff9cebc43c6933eb6147b391147)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:35.247987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2add628-3b31-4ff2-a4ac-3cd5ddeb197b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:35.251863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 166/330] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:35 2022] GET /identity/v3/projects/e5580ff9cebc43c6933eb6147b391147 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:35.255950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2add628-3b31-4ff2-a4ac-3cd5ddeb197b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:35.291636 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d59f9-c8ce-404e-b628-d7ef8fe34fc9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:35.332748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c20694-19bc-49ff-afc7-7b2d1426b445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:35.333312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c20694-19bc-49ff-afc7-7b2d1426b445 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:35.333596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c20694-19bc-49ff-afc7-7b2d1426b445 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:35.333871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c20694-19bc-49ff-afc7-7b2d1426b445 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:35.352434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:35.373222 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2add628-3b31-4ff2-a4ac-3cd5ddeb197b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:35.408345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cb34e-e1f2-454f-a4aa-e24dd880e5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:35.409271 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cb34e-e1f2-454f-a4aa-e24dd880e5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:35.409559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cb34e-e1f2-454f-a4aa-e24dd880e5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:35.409839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cb34e-e1f2-454f-a4aa-e24dd880e5b1 None admin] PATH_INFO: `/v3/groups/d1a083a661a64292a236919e376ef23b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:35.410538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c20694-19bc-49ff-afc7-7b2d1426b445 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:35.410940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cb34e-e1f2-454f-a4aa-e24dd880e5b1 None admin] RBAC: Authorizing `identity:get_group(group_id=d1a083a661a64292a236919e376ef23b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:35.411991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c20694-19bc-49ff-afc7-7b2d1426b445 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:35.412369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cb34e-e1f2-454f-a4aa-e24dd880e5b1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:35.413973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 167/331] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 12:43:35 2022] GET /identity/v3/groups/d1a083a661a64292a236919e376ef23b => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:35.420589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cb34e-e1f2-454f-a4aa-e24dd880e5b1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:35.472984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:35.511671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 165/332] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 15 12:43:35.517696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c20694-19bc-49ff-afc7-7b2d1426b445 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:35.517971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cb34e-e1f2-454f-a4aa-e24dd880e5b1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:35.553283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ac010-0779-4fbe-a103-321201491c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:35.553897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ac010-0779-4fbe-a103-321201491c1f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:35.554175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ac010-0779-4fbe-a103-321201491c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:35.554451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ac010-0779-4fbe-a103-321201491c1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:35.555196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ac010-0779-4fbe-a103-321201491c1f None admin] RBAC: Adding query filter params (group.id=d1a083a661a64292a236919e376ef23b, role.id=ffd57ebcb27d45ef9b9d642f409cc6d7, scope.project.id=e5580ff9cebc43c6933eb6147b391147) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:35.555491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ac010-0779-4fbe-a103-321201491c1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:35.556922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ac010-0779-4fbe-a103-321201491c1f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:35.563438 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 168/333] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 12:43:35 2022] GET /identity/v3/role_assignments?group.id=d1a083a661a64292a236919e376ef23b&scope.project.id=e5580ff9cebc43c6933eb6147b391147&role.id=ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:35.620210 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c20694-19bc-49ff-afc7-7b2d1426b445 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:35.656168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843a761-b2de-46a6-acc4-b46eee730f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:35.656813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843a761-b2de-46a6-acc4-b46eee730f7f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:35.657089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843a761-b2de-46a6-acc4-b46eee730f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:35.657388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843a761-b2de-46a6-acc4-b46eee730f7f None admin] PATH_INFO: `/v3/projects/90479a5430a046cea76f80fced5060d9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:35.658526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843a761-b2de-46a6-acc4-b46eee730f7f None admin] RBAC: Authorizing `identity:get_project(project_id=90479a5430a046cea76f80fced5060d9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:35.660234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843a761-b2de-46a6-acc4-b46eee730f7f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:35.661942 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 166/334] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:43:35 2022] GET /identity/v3/projects/90479a5430a046cea76f80fced5060d9 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:35.667775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ac010-0779-4fbe-a103-321201491c1f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:35.688506 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:35.773096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ac010-0779-4fbe-a103-321201491c1f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:35.806754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabe911-aece-446e-ac67-790e62987349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:35.807392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabe911-aece-446e-ac67-790e62987349 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:35.807665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabe911-aece-446e-ac67-790e62987349 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:35.807937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabe911-aece-446e-ac67-790e62987349 None admin] PATH_INFO: `/v3/groups/46f7c5409b014784a6d5f181d49435ea` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:35.809135 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabe911-aece-446e-ac67-790e62987349 None admin] RBAC: Authorizing `identity:get_group(group_id=46f7c5409b014784a6d5f181d49435ea)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:35.810592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabe911-aece-446e-ac67-790e62987349 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:35.812318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 169/335] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 12:43:35 2022] GET /identity/v3/groups/46f7c5409b014784a6d5f181d49435ea => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:35.818173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843a761-b2de-46a6-acc4-b46eee730f7f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:35.850007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:35.921905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843a761-b2de-46a6-acc4-b46eee730f7f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:35.955502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fb8d3-820b-4b26-8632-e0bc9d684a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:35.956113 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb8d3-820b-4b26-8632-e0bc9d684a96 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:35.956396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb8d3-820b-4b26-8632-e0bc9d684a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:35.956673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb8d3-820b-4b26-8632-e0bc9d684a96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:35.957417 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fb8d3-820b-4b26-8632-e0bc9d684a96 None admin] RBAC: Adding query filter params (group.id=46f7c5409b014784a6d5f181d49435ea, role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.project.id=90479a5430a046cea76f80fced5060d9) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:35.957727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fb8d3-820b-4b26-8632-e0bc9d684a96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:35.959176 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fb8d3-820b-4b26-8632-e0bc9d684a96 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:35.965424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 167/336] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 12:43:35 2022] GET /identity/v3/role_assignments?group.id=46f7c5409b014784a6d5f181d49435ea&scope.project.id=90479a5430a046cea76f80fced5060d9&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:38.899321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6282-e50e-4e7a-b540-161aa80ae43e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:38.899811 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6282-e50e-4e7a-b540-161aa80ae43e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:38.899811 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6282-e50e-4e7a-b540-161aa80ae43e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:38.901425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 170/337] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:38.904411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7720e2-d9e6-4ae2-a124-5230ad869583 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:38.904814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7720e2-d9e6-4ae2-a124-5230ad869583 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:38.905035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7720e2-d9e6-4ae2-a124-5230ad869583 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:38.906625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfc4ed-2abc-486c-8144-f3ebb404ece9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:38.906738 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 168/338] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:38.906961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfc4ed-2abc-486c-8144-f3ebb404ece9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:38.907245 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfc4ed-2abc-486c-8144-f3ebb404ece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:38.911670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5163fb-f1b1-4531-9445-0333ebaa8db3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:38.911954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5163fb-f1b1-4531-9445-0333ebaa8db3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:38.912230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5163fb-f1b1-4531-9445-0333ebaa8db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:38.927122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1edfc4ed-2abc-486c-8144-f3ebb404ece9 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:38.928422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1edfc4ed-2abc-486c-8144-f3ebb404ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:38.942212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9a5163fb-f1b1-4531-9445-0333ebaa8db3 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:38.943421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9a5163fb-f1b1-4531-9445-0333ebaa8db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:39.044333 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 171/339] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:39.067788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 169/340] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:39.127020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bc43d-0f13-447e-99a5-f7f1ce0abf07 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:39.127335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bc43d-0f13-447e-99a5-f7f1ce0abf07 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:39.127577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bc43d-0f13-447e-99a5-f7f1ce0abf07 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:39.128937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 172/341] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:39.134234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8081db-95fe-483c-9366-33cc25a978a5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:39.134521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8081db-95fe-483c-9366-33cc25a978a5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:39.134797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8081db-95fe-483c-9366-33cc25a978a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:39.161687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0b8081db-95fe-483c-9366-33cc25a978a5 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:39.162789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0b8081db-95fe-483c-9366-33cc25a978a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:39.185705 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:39.243514 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 170/342] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:39.382457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bc43d-0f13-447e-99a5-f7f1ce0abf07 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:39.399913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:39.447388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8081db-95fe-483c-9366-33cc25a978a5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:39.522142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bc43d-0f13-447e-99a5-f7f1ce0abf07 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:39.578720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53890d38-ca8f-415f-ae59-b566c8d071dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:39.579322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890d38-ca8f-415f-ae59-b566c8d071dc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:39.579605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890d38-ca8f-415f-ae59-b566c8d071dc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:39.579878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890d38-ca8f-415f-ae59-b566c8d071dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:39.592656 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53890d38-ca8f-415f-ae59-b566c8d071dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:39.594785 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53890d38-ca8f-415f-ae59-b566c8d071dc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:39.611048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8081db-95fe-483c-9366-33cc25a978a5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-53890d38-ca8f-415f-ae59-b566c8d071dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:39.614272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:39.615209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:39.616055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:39.616055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:39.616055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:39.616055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 173/343] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:39.616723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464004a1-1974-4dcc-ae94-b6a8ce6d4965 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:39.616884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464004a1-1974-4dcc-ae94-b6a8ce6d4965 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:39.617118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464004a1-1974-4dcc-ae94-b6a8ce6d4965 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:39.618520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 174/344] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:39.623773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464004a1-1974-4dcc-ae94-b6a8ce6d4965 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:39.656254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badaa9b-b90d-45d7-b891-39ca851729cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:39.657159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badaa9b-b90d-45d7-b891-39ca851729cf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:39.657471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badaa9b-b90d-45d7-b891-39ca851729cf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:39.657743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badaa9b-b90d-45d7-b891-39ca851729cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:39.669078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badaa9b-b90d-45d7-b891-39ca851729cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:39.670562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badaa9b-b90d-45d7-b891-39ca851729cf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2badaa9b-b90d-45d7-b891-39ca851729cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:39.686691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:39.687596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:39.688447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:39.688447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:39.688447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:39.688447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 171/345] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:39.693917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd3f51-8121-42fe-8f36-b0719b0fd669 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:39.694204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd3f51-8121-42fe-8f36-b0719b0fd669 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:39.694481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd3f51-8121-42fe-8f36-b0719b0fd669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:39.699413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:39.707237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:39.712521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b3fd3f51-8121-42fe-8f36-b0719b0fd669 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:39.713727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b3fd3f51-8121-42fe-8f36-b0719b0fd669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:39.739872 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464004a1-1974-4dcc-ae94-b6a8ce6d4965 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:39.778049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a288fc-7458-4ed3-9445-d467e415910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:39.778659 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a288fc-7458-4ed3-9445-d467e415910e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:39.778936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a288fc-7458-4ed3-9445-d467e415910e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:39.779204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a288fc-7458-4ed3-9445-d467e415910e None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:39.779844 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a288fc-7458-4ed3-9445-d467e415910e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:39.780151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a288fc-7458-4ed3-9445-d467e415910e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:39.789436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 172/346] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:39.789566 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a288fc-7458-4ed3-9445-d467e415910e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:39.789649 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd3f51-8121-42fe-8f36-b0719b0fd669 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:39.802131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 175/347] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:39.803044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a288fc-7458-4ed3-9445-d467e415910e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:39.920513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd3f51-8121-42fe-8f36-b0719b0fd669 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:39.921365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a288fc-7458-4ed3-9445-d467e415910e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:39.951704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:39.957999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4bbd7-0648-47ed-a0a1-6069596a087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:39.958834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4bbd7-0648-47ed-a0a1-6069596a087e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:39.959334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4bbd7-0648-47ed-a0a1-6069596a087e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:39.959754 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4bbd7-0648-47ed-a0a1-6069596a087e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:39.961089 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc51ae-8eb2-4a2c-bbdc-f4010ee52fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:39.964440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc51ae-8eb2-4a2c-bbdc-f4010ee52fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:39.964440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc51ae-8eb2-4a2c-bbdc-f4010ee52fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:39.964440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc51ae-8eb2-4a2c-bbdc-f4010ee52fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:39.964780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:39.964934 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc51ae-8eb2-4a2c-bbdc-f4010ee52fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:39.964934 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc51ae-8eb2-4a2c-bbdc-f4010ee52fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:39.965706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc51ae-8eb2-4a2c-bbdc-f4010ee52fb8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:39.970855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4bbd7-0648-47ed-a0a1-6069596a087e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:39.972300 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4bbd7-0648-47ed-a0a1-6069596a087e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:39.977980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 176/348] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:39.978934 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc51ae-8eb2-4a2c-bbdc-f4010ee52fb8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9dc4bbd7-0648-47ed-a0a1-6069596a087e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:39.989136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:39.993968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:39.994827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:39.994827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:39.994827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:39.994827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 173/349] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:39.994827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4bbd7-0648-47ed-a0a1-6069596a087e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:40.097998 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc51ae-8eb2-4a2c-bbdc-f4010ee52fb8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:40.120668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4bbd7-0648-47ed-a0a1-6069596a087e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:40.138787 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4fd60-0303-408f-bac9-f0ef6f97bfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:40.139754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4fd60-0303-408f-bac9-f0ef6f97bfab None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:40.140443 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4fd60-0303-408f-bac9-f0ef6f97bfab None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:40.140885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4fd60-0303-408f-bac9-f0ef6f97bfab None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:40.141782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4fd60-0303-408f-bac9-f0ef6f97bfab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:40.143794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4fd60-0303-408f-bac9-f0ef6f97bfab None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:40.157323 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2489735-da2a-4030-96e7-af27ef5aa038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:40.157934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2489735-da2a-4030-96e7-af27ef5aa038 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:40.158215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2489735-da2a-4030-96e7-af27ef5aa038 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:40.158487 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2489735-da2a-4030-96e7-af27ef5aa038 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:40.159399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2489735-da2a-4030-96e7-af27ef5aa038 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:40.160823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2489735-da2a-4030-96e7-af27ef5aa038 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:40.217577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 177/350] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:39 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:40.218333 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4fd60-0303-408f-bac9-f0ef6f97bfab None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:40.248285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 174/351] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:39 2022] POST /identity/v3/users => generated 246 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:40.249654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2489735-da2a-4030-96e7-af27ef5aa038 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:40.282316 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:40.326848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:40.369714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4fd60-0303-408f-bac9-f0ef6f97bfab None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:40.416467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2489735-da2a-4030-96e7-af27ef5aa038 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:40.422452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6641b-cb88-4b3f-95a3-3d296f23a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:40.423089 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6641b-cb88-4b3f-95a3-3d296f23a80a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:40.423368 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6641b-cb88-4b3f-95a3-3d296f23a80a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:40.423643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6641b-cb88-4b3f-95a3-3d296f23a80a None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:40.424298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6641b-cb88-4b3f-95a3-3d296f23a80a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:40.424604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6641b-cb88-4b3f-95a3-3d296f23a80a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:40.426040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6641b-cb88-4b3f-95a3-3d296f23a80a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:40.441310 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 178/352] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:40.442408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a56c20-ab7c-400b-a7e9-51b4c9eac8b5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:40.442694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a56c20-ab7c-400b-a7e9-51b4c9eac8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:40.442966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a56c20-ab7c-400b-a7e9-51b4c9eac8b5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:40.444420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 179/353] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:40.447954 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a56c20-ab7c-400b-a7e9-51b4c9eac8b5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:40.470639 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d1f45-3bc7-4370-9210-681f1fe1627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:40.471211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d1f45-3bc7-4370-9210-681f1fe1627b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:40.471497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d1f45-3bc7-4370-9210-681f1fe1627b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:40.471775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d1f45-3bc7-4370-9210-681f1fe1627b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:40.486094 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d1f45-3bc7-4370-9210-681f1fe1627b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:40.493246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d1f45-3bc7-4370-9210-681f1fe1627b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e02d1f45-3bc7-4370-9210-681f1fe1627b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:40.502792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:40.503717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:40.504999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:40.504999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:40.504999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:40.504999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 175/354] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:40.505438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eb143-09fc-44b3-8fb7-8acb3981ec5b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:40.506140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eb143-09fc-44b3-8fb7-8acb3981ec5b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:40.506140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eb143-09fc-44b3-8fb7-8acb3981ec5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:40.532196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-392eb143-09fc-44b3-8fb7-8acb3981ec5b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:40.535137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-392eb143-09fc-44b3-8fb7-8acb3981ec5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:40.592137 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a56c20-ab7c-400b-a7e9-51b4c9eac8b5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:40.594364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:40.628787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 176/355] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:40.630016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eb143-09fc-44b3-8fb7-8acb3981ec5b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:40.641133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1683f-6d9c-4873-8e73-9e19b7170de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:40.642726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1683f-6d9c-4873-8e73-9e19b7170de8 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:40.643009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1683f-6d9c-4873-8e73-9e19b7170de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:40.647810 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1683f-6d9c-4873-8e73-9e19b7170de8 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:40.648488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1683f-6d9c-4873-8e73-9e19b7170de8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:40.649991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1683f-6d9c-4873-8e73-9e19b7170de8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:40.680883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:40.755893 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eb143-09fc-44b3-8fb7-8acb3981ec5b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:40.795403 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 180/356] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:40 2022] POST /identity/v3/users => generated 248 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:40.799518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22423a45-32b0-40a3-bdc8-6c574e9b2491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:40.800161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22423a45-32b0-40a3-bdc8-6c574e9b2491 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:40.800443 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22423a45-32b0-40a3-bdc8-6c574e9b2491 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:40.800724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22423a45-32b0-40a3-bdc8-6c574e9b2491 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:40.801482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22423a45-32b0-40a3-bdc8-6c574e9b2491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:40.801798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22423a45-32b0-40a3-bdc8-6c574e9b2491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:40.803364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22423a45-32b0-40a3-bdc8-6c574e9b2491 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:40.829540 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 177/357] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:40.829816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1683f-6d9c-4873-8e73-9e19b7170de8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:40.912179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22423a45-32b0-40a3-bdc8-6c574e9b2491 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:40.982585 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1683f-6d9c-4873-8e73-9e19b7170de8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:41.049549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9425af-249f-4565-81e6-6ab9b40a8e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:41.050175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9425af-249f-4565-81e6-6ab9b40a8e71 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:41.050461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9425af-249f-4565-81e6-6ab9b40a8e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:41.050741 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9425af-249f-4565-81e6-6ab9b40a8e71 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:41.051402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9425af-249f-4565-81e6-6ab9b40a8e71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:41.051833 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:41.052899 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9425af-249f-4565-81e6-6ab9b40a8e71 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:41.068683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22423a45-32b0-40a3-bdc8-6c574e9b2491 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:41.111997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:41.112622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:41.112904 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:41.113179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:41.113935 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:41.115331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:41.149548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 181/358] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:40 2022] POST /identity/v3/users => generated 249 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:41.194571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.sql.core [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 12:43:41.194571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 15 12:43:41.194571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [parameters: {'id': 'c8568609c1c249c1b6f850d41cbba68f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 15 12:43:41.194571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81613) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 15 12:43:41.217640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return ref.to_dict() Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:43:41.218598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application yield resource Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise self.value Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application session.commit() Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 12:43:41.220523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.flush() Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application compat.raise_( Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 12:43:41.222638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:43:41.224759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:43:41.226757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [parameters: {'id': 'c8568609c1c249c1b6f850d41cbba68f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:41.228722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 15 12:43:41.234829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:41.236642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 178/359] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 12:43:40 2022] POST /identity/v3/roles => generated 144 bytes in 307 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:41.236833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9425af-249f-4565-81e6-6ab9b40a8e71 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:41.268916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:41.392964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9425af-249f-4565-81e6-6ab9b40a8e71 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:41.432409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25642db-1d10-45b2-b60d-61772ba787c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:41.433054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25642db-1d10-45b2-b60d-61772ba787c1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:41.433963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25642db-1d10-45b2-b60d-61772ba787c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:41.434246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25642db-1d10-45b2-b60d-61772ba787c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:41.444502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25642db-1d10-45b2-b60d-61772ba787c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d25642db-1d10-45b2-b60d-61772ba787c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:41.450033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:41.452177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:41.454258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 182/360] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:43:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:41.463996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:41.623692 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc185f98-b19a-496e-b3c2-6b0d259c3aef None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:41.674336 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355af97-1c0e-4c91-b8dc-829923b391b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:41.674882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355af97-1c0e-4c91-b8dc-829923b391b0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:41.675162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355af97-1c0e-4c91-b8dc-829923b391b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:41.675440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355af97-1c0e-4c91-b8dc-829923b391b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:41.676105 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355af97-1c0e-4c91-b8dc-829923b391b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:41.676409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355af97-1c0e-4c91-b8dc-829923b391b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:41.677819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355af97-1c0e-4c91-b8dc-829923b391b0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:41.692964 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 179/361] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:42.676877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8af507-0e9a-46e4-bb80-fbfdc5aa0a36 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:42.677749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8af507-0e9a-46e4-bb80-fbfdc5aa0a36 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:42.678201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8af507-0e9a-46e4-bb80-fbfdc5aa0a36 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:42.679919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 183/362] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:42.685665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180c1fb-50d8-45e1-aa7e-7b8df90e2a18 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:42.686427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180c1fb-50d8-45e1-aa7e-7b8df90e2a18 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:42.686869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180c1fb-50d8-45e1-aa7e-7b8df90e2a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:42.709613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a180c1fb-50d8-45e1-aa7e-7b8df90e2a18 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:42.711023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a180c1fb-50d8-45e1-aa7e-7b8df90e2a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:42.723976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:42.805330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 180/363] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:43.020424 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84844569-ed64-404d-8e88-93af32c0ee74 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.020919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84844569-ed64-404d-8e88-93af32c0ee74 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.021400 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84844569-ed64-404d-8e88-93af32c0ee74 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.023123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 184/364] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:43.029588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3d08e-f53c-49aa-a403-80c02a37271e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.030054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3d08e-f53c-49aa-a403-80c02a37271e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.030490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3d08e-f53c-49aa-a403-80c02a37271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.049074 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e0e3d08e-f53c-49aa-a403-80c02a37271e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:43.050496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e0e3d08e-f53c-49aa-a403-80c02a37271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:43.126790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 181/365] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:43.181446 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84844569-ed64-404d-8e88-93af32c0ee74 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:43.269725 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:43.296324 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84844569-ed64-404d-8e88-93af32c0ee74 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:43.331952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c4e90-ac47-44d6-9b57-653fe92374e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:43.332822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c4e90-ac47-44d6-9b57-653fe92374e7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.333364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c4e90-ac47-44d6-9b57-653fe92374e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.333778 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c4e90-ac47-44d6-9b57-653fe92374e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.344737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c4e90-ac47-44d6-9b57-653fe92374e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1b2c4e90-ac47-44d6-9b57-653fe92374e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:43.350476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:43.351420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:43.352239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:43.354810 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 185/366] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:43.362512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3d08e-f53c-49aa-a403-80c02a37271e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:43.458343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:43.494042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3d08e-f53c-49aa-a403-80c02a37271e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:43.544876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c4e90-ac47-44d6-9b57-653fe92374e7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:43.556581 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8cdb1-0662-4b5a-9366-359549d53672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:43.557397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8cdb1-0662-4b5a-9366-359549d53672 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.557510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8cdb1-0662-4b5a-9366-359549d53672 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.557688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8cdb1-0662-4b5a-9366-359549d53672 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.558466 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8cdb1-0662-4b5a-9366-359549d53672 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:43.558783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8cdb1-0662-4b5a-9366-359549d53672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:43.560413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8cdb1-0662-4b5a-9366-359549d53672 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:43.575027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 182/367] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:43.580216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8cdb1-0662-4b5a-9366-359549d53672 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:43.710008 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c4e90-ac47-44d6-9b57-653fe92374e7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:43.764848 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bca6f-5a96-4d1b-bcb9-f31c66c45a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:43.765548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bca6f-5a96-4d1b-bcb9-f31c66c45a7a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.765824 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bca6f-5a96-4d1b-bcb9-f31c66c45a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.766092 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bca6f-5a96-4d1b-bcb9-f31c66c45a7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.767967 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:43.773510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8cdb1-0662-4b5a-9366-359549d53672 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:43.785228 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bca6f-5a96-4d1b-bcb9-f31c66c45a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-076bca6f-5a96-4d1b-bcb9-f31c66c45a7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:43.788106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:43.788941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 186/368] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:43.790116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9518b6e-d3a8-43ec-ba3d-9ad6d290160e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.790827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9518b6e-d3a8-43ec-ba3d-9ad6d290160e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.790827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9518b6e-d3a8-43ec-ba3d-9ad6d290160e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.796808 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 187/369] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:43.796808 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9518b6e-d3a8-43ec-ba3d-9ad6d290160e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:43.825976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cab8d-6767-418f-8728-6db2486f3bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:43.826788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cab8d-6767-418f-8728-6db2486f3bcc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.827084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cab8d-6767-418f-8728-6db2486f3bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.827363 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cab8d-6767-418f-8728-6db2486f3bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.829832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:43.839024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cab8d-6767-418f-8728-6db2486f3bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:43.841690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cab8d-6767-418f-8728-6db2486f3bcc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c23cab8d-6767-418f-8728-6db2486f3bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:43.857510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:43.858458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:43.859392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:43.859392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:43.859392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:43.859392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 183/370] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:43.860008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cf684-73e2-4e07-911e-7839a9b1ec91 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.860008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cf684-73e2-4e07-911e-7839a9b1ec91 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.860293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cf684-73e2-4e07-911e-7839a9b1ec91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.879002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-144cf684-73e2-4e07-911e-7839a9b1ec91 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:43.880212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-144cf684-73e2-4e07-911e-7839a9b1ec91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:43.915429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9518b6e-d3a8-43ec-ba3d-9ad6d290160e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:43.961220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 184/371] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:43.961435 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e3749-fb9b-4cae-bcd2-d3a3d867fdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:43.961435 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e3749-fb9b-4cae-bcd2-d3a3d867fdc4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.961435 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e3749-fb9b-4cae-bcd2-d3a3d867fdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.961672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7168c-3a28-4131-bc94-7a9bee0787a3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.961750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e3749-fb9b-4cae-bcd2-d3a3d867fdc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.961836 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7168c-3a28-4131-bc94-7a9bee0787a3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.961836 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7168c-3a28-4131-bc94-7a9bee0787a3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.961978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e3749-fb9b-4cae-bcd2-d3a3d867fdc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:43.961978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e3749-fb9b-4cae-bcd2-d3a3d867fdc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:43.961978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e3749-fb9b-4cae-bcd2-d3a3d867fdc4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:43.962185 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 185/372] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:43.962185 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7168c-3a28-4131-bc94-7a9bee0787a3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:43.972775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 188/373] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:43.974249 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9904e4c-db75-4f70-8989-7812f8aa5bbd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:43.974641 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9904e4c-db75-4f70-8989-7812f8aa5bbd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:43.975036 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9904e4c-db75-4f70-8989-7812f8aa5bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:43.994416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b9904e4c-db75-4f70-8989-7812f8aa5bbd None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:43.996338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b9904e4c-db75-4f70-8989-7812f8aa5bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:44.052980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:44.075479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 189/374] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:44.076755 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9904e4c-db75-4f70-8989-7812f8aa5bbd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.107109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7168c-3a28-4131-bc94-7a9bee0787a3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:44.138838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:44.173811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7cf08-abfa-4f82-a021-5bec3b110741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:44.175938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7cf08-abfa-4f82-a021-5bec3b110741 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.176425 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7cf08-abfa-4f82-a021-5bec3b110741 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.176930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7cf08-abfa-4f82-a021-5bec3b110741 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.177806 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7cf08-abfa-4f82-a021-5bec3b110741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:44.178469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7cf08-abfa-4f82-a021-5bec3b110741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:44.183524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7cf08-abfa-4f82-a021-5bec3b110741 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:44.203742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 186/375] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:44.210172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7cf08-abfa-4f82-a021-5bec3b110741 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.277889 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9904e4c-db75-4f70-8989-7812f8aa5bbd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:44.319161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090c374-197b-44dc-9969-7e3b0913a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:44.320003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090c374-197b-44dc-9969-7e3b0913a380 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.320336 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090c374-197b-44dc-9969-7e3b0913a380 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.320612 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090c374-197b-44dc-9969-7e3b0913a380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.335344 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090c374-197b-44dc-9969-7e3b0913a380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:44.338169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090c374-197b-44dc-9969-7e3b0913a380 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3090c374-197b-44dc-9969-7e3b0913a380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:44.356615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:44.357581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:44.358406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:44.358406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:44.358406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:44.362543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 190/376] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:44.363315 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090c374-197b-44dc-9969-7e3b0913a380 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.364777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7cf08-abfa-4f82-a021-5bec3b110741 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:44.404189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a13c72-636c-4a3b-b2e5-41a152a091a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:44.404818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a13c72-636c-4a3b-b2e5-41a152a091a5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.405299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:44.405551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a13c72-636c-4a3b-b2e5-41a152a091a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.405831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a13c72-636c-4a3b-b2e5-41a152a091a5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.414537 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a13c72-636c-4a3b-b2e5-41a152a091a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:44.415987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a13c72-636c-4a3b-b2e5-41a152a091a5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-40a13c72-636c-4a3b-b2e5-41a152a091a5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:44.430677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:44.431677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:44.432602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 187/377] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:44.434791 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a13c72-636c-4a3b-b2e5-41a152a091a5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.474730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:44.478319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090c374-197b-44dc-9969-7e3b0913a380 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:44.515710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb64244-5c1d-44f9-af39-da5a2b314f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:44.516715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb64244-5c1d-44f9-af39-da5a2b314f2d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.517168 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb64244-5c1d-44f9-af39-da5a2b314f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.517749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb64244-5c1d-44f9-af39-da5a2b314f2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.530014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb64244-5c1d-44f9-af39-da5a2b314f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cdb64244-5c1d-44f9-af39-da5a2b314f2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:44.536252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:44.537208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:44.538013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:44.542328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 191/378] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:44.543833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb64244-5c1d-44f9-af39-da5a2b314f2d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.546419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a13c72-636c-4a3b-b2e5-41a152a091a5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:44.581712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56e5af-d46e-4d95-a7f3-3bdef14ee1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:44.582327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56e5af-d46e-4d95-a7f3-3bdef14ee1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.582599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56e5af-d46e-4d95-a7f3-3bdef14ee1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.582868 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56e5af-d46e-4d95-a7f3-3bdef14ee1e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.583498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56e5af-d46e-4d95-a7f3-3bdef14ee1e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:44.583797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56e5af-d46e-4d95-a7f3-3bdef14ee1e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:44.585178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56e5af-d46e-4d95-a7f3-3bdef14ee1e7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:44.596605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 188/379] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:44.597375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56e5af-d46e-4d95-a7f3-3bdef14ee1e7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.645706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:44.649404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb64244-5c1d-44f9-af39-da5a2b314f2d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:44.683957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e6666-1e5d-47fb-9c0d-eb793496d60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:44.684589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e6666-1e5d-47fb-9c0d-eb793496d60f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.684864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e6666-1e5d-47fb-9c0d-eb793496d60f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.685132 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e6666-1e5d-47fb-9c0d-eb793496d60f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.695121 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e6666-1e5d-47fb-9c0d-eb793496d60f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d05e6666-1e5d-47fb-9c0d-eb793496d60f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:44.700977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:44.701898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:44.702698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:44.703308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:44.707223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 192/380] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:44.707805 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56e5af-d46e-4d95-a7f3-3bdef14ee1e7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:44.708070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e6666-1e5d-47fb-9c0d-eb793496d60f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.741804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da901f-f7fc-43cf-81da-c6d0b4ad81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:44.742412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da901f-f7fc-43cf-81da-c6d0b4ad81f2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.742693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da901f-f7fc-43cf-81da-c6d0b4ad81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.742965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da901f-f7fc-43cf-81da-c6d0b4ad81f2 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.743684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da901f-f7fc-43cf-81da-c6d0b4ad81f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:44.743990 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da901f-f7fc-43cf-81da-c6d0b4ad81f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:44.745479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da901f-f7fc-43cf-81da-c6d0b4ad81f2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:44.780841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 189/381] 10.222.0.30 () {56 vars in 1137 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:44.782106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2eef85-6b7f-4593-925d-15f759e5e3af None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.782389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2eef85-6b7f-4593-925d-15f759e5e3af None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.782666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2eef85-6b7f-4593-925d-15f759e5e3af None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.784070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 190/382] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:44.784849 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2eef85-6b7f-4593-925d-15f759e5e3af None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.812201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e6666-1e5d-47fb-9c0d-eb793496d60f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:44.845556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc57bf-88be-48b9-a3f9-d412ef10a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:44.846159 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc57bf-88be-48b9-a3f9-d412ef10a01e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.846434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc57bf-88be-48b9-a3f9-d412ef10a01e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.846755 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc57bf-88be-48b9-a3f9-d412ef10a01e None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.847360 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc57bf-88be-48b9-a3f9-d412ef10a01e None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:44.847660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc57bf-88be-48b9-a3f9-d412ef10a01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:44.849048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc57bf-88be-48b9-a3f9-d412ef10a01e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:44.857563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 193/383] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:44.858341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc57bf-88be-48b9-a3f9-d412ef10a01e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.884290 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:44.888130 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2eef85-6b7f-4593-925d-15f759e5e3af None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:44.921922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29d93b-17f2-46a1-9092-c179f23a2b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:44.922541 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29d93b-17f2-46a1-9092-c179f23a2b7a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:44.922817 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29d93b-17f2-46a1-9092-c179f23a2b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:44.923088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29d93b-17f2-46a1-9092-c179f23a2b7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:44.929898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29d93b-17f2-46a1-9092-c179f23a2b7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:44.931536 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29d93b-17f2-46a1-9092-c179f23a2b7a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7e29d93b-17f2-46a1-9092-c179f23a2b7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:44.943597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:44.944558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 191/384] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:44.945592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29d93b-17f2-46a1-9092-c179f23a2b7a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:44.960596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:44.967494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc57bf-88be-48b9-a3f9-d412ef10a01e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.002292 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff3605-dbb1-4914-8261-5b0685cf369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.002903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff3605-dbb1-4914-8261-5b0685cf369c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.003175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff3605-dbb1-4914-8261-5b0685cf369c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.003445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff3605-dbb1-4914-8261-5b0685cf369c None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.004101 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff3605-dbb1-4914-8261-5b0685cf369c None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:45.004403 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff3605-dbb1-4914-8261-5b0685cf369c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.005804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff3605-dbb1-4914-8261-5b0685cf369c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.014249 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 194/385] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:45.015172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60165ee1-cb76-4abc-bbd6-deafe49f6d30 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.015459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60165ee1-cb76-4abc-bbd6-deafe49f6d30 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.015728 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60165ee1-cb76-4abc-bbd6-deafe49f6d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.035102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-60165ee1-cb76-4abc-bbd6-deafe49f6d30 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:45.036273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-60165ee1-cb76-4abc-bbd6-deafe49f6d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:45.045983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29d93b-17f2-46a1-9092-c179f23a2b7a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.079503 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22103a-323d-4f04-963c-4fca380d81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.080131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22103a-323d-4f04-963c-4fca380d81f6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.080410 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22103a-323d-4f04-963c-4fca380d81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.080687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22103a-323d-4f04-963c-4fca380d81f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.091204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22103a-323d-4f04-963c-4fca380d81f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.092678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22103a-323d-4f04-963c-4fca380d81f6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.102530 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 195/386] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:45.103306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60165ee1-cb76-4abc-bbd6-deafe49f6d30 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-3f22103a-323d-4f04-963c-4fca380d81f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:45.108739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:45.109734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:45.110614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:45.110614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.110614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:45.110614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 192/387] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:45.110614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22103a-323d-4f04-963c-4fca380d81f6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.119428 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:45.178578 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:45.215597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60165ee1-cb76-4abc-bbd6-deafe49f6d30 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.218606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22103a-323d-4f04-963c-4fca380d81f6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.250503 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b0867-f5a6-4bc1-a355-ab75d4c61749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.251136 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b0867-f5a6-4bc1-a355-ab75d4c61749 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.251413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b0867-f5a6-4bc1-a355-ab75d4c61749 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.251680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b0867-f5a6-4bc1-a355-ab75d4c61749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.254228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c4ab6-5eaa-41f9-96f7-c612fcdad4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.254933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c4ab6-5eaa-41f9-96f7-c612fcdad4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.255219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c4ab6-5eaa-41f9-96f7-c612fcdad4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.255484 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c4ab6-5eaa-41f9-96f7-c612fcdad4b5 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.256209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c4ab6-5eaa-41f9-96f7-c612fcdad4b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:45.256518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c4ab6-5eaa-41f9-96f7-c612fcdad4b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.258096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c4ab6-5eaa-41f9-96f7-c612fcdad4b5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.261906 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b0867-f5a6-4bc1-a355-ab75d4c61749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.263476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b0867-f5a6-4bc1-a355-ab75d4c61749 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.280883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 193/388] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:45.280883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c4ab6-5eaa-41f9-96f7-c612fcdad4b5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e22b0867-f5a6-4bc1-a355-ab75d4c61749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:45.281375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:45.282409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:45.283353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:45.283353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.283353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:45.283353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 196/389] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:45.283353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b0867-f5a6-4bc1-a355-ab75d4c61749 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.352294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:45.387142 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c4ab6-5eaa-41f9-96f7-c612fcdad4b5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.393312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b0867-f5a6-4bc1-a355-ab75d4c61749 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.411322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:45.421804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a75f4-74f3-4e91-85d6-1bbd2acd5415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.422442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a75f4-74f3-4e91-85d6-1bbd2acd5415 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.422781 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a75f4-74f3-4e91-85d6-1bbd2acd5415 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.423059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a75f4-74f3-4e91-85d6-1bbd2acd5415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.425646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e4793-4d44-470e-8869-2b74b772efe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.426259 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4793-4d44-470e-8869-2b74b772efe0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.426540 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4793-4d44-470e-8869-2b74b772efe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.426817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4793-4d44-470e-8869-2b74b772efe0 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.427460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e4793-4d44-470e-8869-2b74b772efe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:45.427765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e4793-4d44-470e-8869-2b74b772efe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.429142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e4793-4d44-470e-8869-2b74b772efe0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.433618 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a75f4-74f3-4e91-85d6-1bbd2acd5415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.435434 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a75f4-74f3-4e91-85d6-1bbd2acd5415 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.441280 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 197/390] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:45.442247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e4793-4d44-470e-8869-2b74b772efe0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-142a75f4-74f3-4e91-85d6-1bbd2acd5415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:45.453978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:45.455076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:45.455958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:45.455958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.455958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:45.455958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 194/391] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:45.456245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a75f4-74f3-4e91-85d6-1bbd2acd5415 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.546712 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e4793-4d44-470e-8869-2b74b772efe0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.556755 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a75f4-74f3-4e91-85d6-1bbd2acd5415 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.570548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:45.581363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923367c5-9ccb-4d12-aa0f-1d550862ba71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.581992 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923367c5-9ccb-4d12-aa0f-1d550862ba71 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.582273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923367c5-9ccb-4d12-aa0f-1d550862ba71 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.582541 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923367c5-9ccb-4d12-aa0f-1d550862ba71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.590412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110b986-dc57-42a5-a072-954c58f78b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.591016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110b986-dc57-42a5-a072-954c58f78b3f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.591296 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110b986-dc57-42a5-a072-954c58f78b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.591572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110b986-dc57-42a5-a072-954c58f78b3f None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.592207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110b986-dc57-42a5-a072-954c58f78b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:45.592511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110b986-dc57-42a5-a072-954c58f78b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.592782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923367c5-9ccb-4d12-aa0f-1d550862ba71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.593943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110b986-dc57-42a5-a072-954c58f78b3f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.594246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923367c5-9ccb-4d12-aa0f-1d550862ba71 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.606141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 195/392] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:45.606959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110b986-dc57-42a5-a072-954c58f78b3f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-923367c5-9ccb-4d12-aa0f-1d550862ba71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:45.609487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:45.610543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:45.611542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:45.611542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.611542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:45.611542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 198/393] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:45.611542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923367c5-9ccb-4d12-aa0f-1d550862ba71 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.648875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:45.716021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923367c5-9ccb-4d12-aa0f-1d550862ba71 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.716247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110b986-dc57-42a5-a072-954c58f78b3f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.748832 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bb5a4-1f61-4be0-9bd1-b923921e738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.749496 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb5a4-1f61-4be0-9bd1-b923921e738e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.749779 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb5a4-1f61-4be0-9bd1-b923921e738e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.750048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb5a4-1f61-4be0-9bd1-b923921e738e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.751613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964a673-bad2-4dc0-93bd-038318840233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.752241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a673-bad2-4dc0-93bd-038318840233 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.752524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a673-bad2-4dc0-93bd-038318840233 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.752806 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a673-bad2-4dc0-93bd-038318840233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.756928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bb5a4-1f61-4be0-9bd1-b923921e738e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.758401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bb5a4-1f61-4be0-9bd1-b923921e738e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.763572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964a673-bad2-4dc0-93bd-038318840233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.765092 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964a673-bad2-4dc0-93bd-038318840233 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-fa2bb5a4-1f61-4be0-9bd1-b923921e738e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:45.770411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:45.771395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 199/394] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:45.772327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bb5a4-1f61-4be0-9bd1-b923921e738e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a964a673-bad2-4dc0-93bd-038318840233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:45.781441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:45.782852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:45.783908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:45.783908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:45.783908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:45.783908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 196/395] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:45.783908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964a673-bad2-4dc0-93bd-038318840233 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.812989 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:45.871177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:45.878285 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bb5a4-1f61-4be0-9bd1-b923921e738e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.882696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964a673-bad2-4dc0-93bd-038318840233 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:45.911713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e364a3a-f102-4464-b810-c07577046f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.912244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e364a3a-f102-4464-b810-c07577046f36 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.912520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e364a3a-f102-4464-b810-c07577046f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.912791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e364a3a-f102-4464-b810-c07577046f36 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.913449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e364a3a-f102-4464-b810-c07577046f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:45.913753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e364a3a-f102-4464-b810-c07577046f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.915108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e364a3a-f102-4464-b810-c07577046f36 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.916152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9241-2b81-480d-849d-994fa49e330e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:45.916777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9241-2b81-480d-849d-994fa49e330e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:45.917056 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9241-2b81-480d-849d-994fa49e330e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:45.917396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9241-2b81-480d-849d-994fa49e330e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:45.924331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9241-2b81-480d-849d-994fa49e330e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:45.925816 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9241-2b81-480d-849d-994fa49e330e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:45.926650 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 200/396] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:45.927267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e364a3a-f102-4464-b810-c07577046f36 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2f9e9241-2b81-480d-849d-994fa49e330e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:45.938511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:45.939598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:45.940721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 197/397] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:45.942235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9241-2b81-480d-849d-994fa49e330e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.024160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:46.030484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e364a3a-f102-4464-b810-c07577046f36 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.042920 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9241-2b81-480d-849d-994fa49e330e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.067558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823c16a-fdd1-4eb1-b76b-e0b477f7fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.068169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823c16a-fdd1-4eb1-b76b-e0b477f7fe27 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.068457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823c16a-fdd1-4eb1-b76b-e0b477f7fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.068733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823c16a-fdd1-4eb1-b76b-e0b477f7fe27 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.069418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823c16a-fdd1-4eb1-b76b-e0b477f7fe27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:46.069726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823c16a-fdd1-4eb1-b76b-e0b477f7fe27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.071095 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823c16a-fdd1-4eb1-b76b-e0b477f7fe27 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.076803 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e6d59-e8b3-471c-96bc-b9cdc1f2e67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.077428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e6d59-e8b3-471c-96bc-b9cdc1f2e67b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.077707 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e6d59-e8b3-471c-96bc-b9cdc1f2e67b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.077979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e6d59-e8b3-471c-96bc-b9cdc1f2e67b None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.078658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e6d59-e8b3-471c-96bc-b9cdc1f2e67b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:46.078959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e6d59-e8b3-471c-96bc-b9cdc1f2e67b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.080370 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e6d59-e8b3-471c-96bc-b9cdc1f2e67b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.083301 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 201/398] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:46.084159 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823c16a-fdd1-4eb1-b76b-e0b477f7fe27 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.100986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 198/399] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:46.101848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e6d59-e8b3-471c-96bc-b9cdc1f2e67b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.112376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:46.188645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823c16a-fdd1-4eb1-b76b-e0b477f7fe27 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.200333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e6d59-e8b3-471c-96bc-b9cdc1f2e67b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.222117 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a71c3-3358-4bad-b5eb-da3318de6b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.222717 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a71c3-3358-4bad-b5eb-da3318de6b4a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.222994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a71c3-3358-4bad-b5eb-da3318de6b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.223263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a71c3-3358-4bad-b5eb-da3318de6b4a None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.223893 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a71c3-3358-4bad-b5eb-da3318de6b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:46.224194 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a71c3-3358-4bad-b5eb-da3318de6b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.225579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a71c3-3358-4bad-b5eb-da3318de6b4a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.232750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4866019-9d01-445e-b4d2-4e3683c3aa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.233518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866019-9d01-445e-b4d2-4e3683c3aa6f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.233798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866019-9d01-445e-b4d2-4e3683c3aa6f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.234066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866019-9d01-445e-b4d2-4e3683c3aa6f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.237722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 202/400] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:46.238497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a71c3-3358-4bad-b5eb-da3318de6b4a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.241487 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4866019-9d01-445e-b4d2-4e3683c3aa6f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.242959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4866019-9d01-445e-b4d2-4e3683c3aa6f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e4866019-9d01-445e-b4d2-4e3683c3aa6f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:46.255139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:46.256221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 199/401] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:46.257476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4866019-9d01-445e-b4d2-4e3683c3aa6f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.258406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:46.324905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:46.359142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a71c3-3358-4bad-b5eb-da3318de6b4a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.360263 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4866019-9d01-445e-b4d2-4e3683c3aa6f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.392753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ce0a8-fcb6-4abb-acba-da4d8b965987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.393381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ce0a8-fcb6-4abb-acba-da4d8b965987 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.393657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ce0a8-fcb6-4abb-acba-da4d8b965987 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.393928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ce0a8-fcb6-4abb-acba-da4d8b965987 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.394139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99216cff-3a0b-44ba-b487-845b4564e29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.394645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ce0a8-fcb6-4abb-acba-da4d8b965987 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:46.394764 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99216cff-3a0b-44ba-b487-845b4564e29b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.394950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ce0a8-fcb6-4abb-acba-da4d8b965987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.395047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99216cff-3a0b-44ba-b487-845b4564e29b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.395311 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99216cff-3a0b-44ba-b487-845b4564e29b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.396373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ce0a8-fcb6-4abb-acba-da4d8b965987 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.401875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99216cff-3a0b-44ba-b487-845b4564e29b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.404007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99216cff-3a0b-44ba-b487-845b4564e29b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-99216cff-3a0b-44ba-b487-845b4564e29b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:46.415938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:46.416905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:46.418246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:46.418246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:46.418246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:46.418246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:46.418246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:46.418246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:46.418246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:46.418246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:46.418246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 200/402] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:46.418823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99216cff-3a0b-44ba-b487-845b4564e29b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.435622 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 203/403] 10.222.0.30 () {56 vars in 1137 bytes} [Sat Oct 15 12:43:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:46.436450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ce0a8-fcb6-4abb-acba-da4d8b965987 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.514244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:46.517094 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99216cff-3a0b-44ba-b487-845b4564e29b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.537975 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ce0a8-fcb6-4abb-acba-da4d8b965987 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.543945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:46.551351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cae9c-8c68-418e-883e-1fac6df8651a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.552007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cae9c-8c68-418e-883e-1fac6df8651a None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.552291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cae9c-8c68-418e-883e-1fac6df8651a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.552562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cae9c-8c68-418e-883e-1fac6df8651a None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/5fc4d28cded84afba73b053d18d4aa23/roles/5a5e7aced6d84267bd9e1fa0c1016420` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.569092 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cae9c-8c68-418e-883e-1fac6df8651a None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=5fc4d28cded84afba73b053d18d4aa23, role_id=5a5e7aced6d84267bd9e1fa0c1016420)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.570619 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cae9c-8c68-418e-883e-1fac6df8651a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.572693 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62192-1db8-4795-8721-05ddbe0cb1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.573334 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62192-1db8-4795-8721-05ddbe0cb1e9 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.573649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62192-1db8-4795-8721-05ddbe0cb1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.573968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62192-1db8-4795-8721-05ddbe0cb1e9 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.574590 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e62192-1db8-4795-8721-05ddbe0cb1e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.576032 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e62192-1db8-4795-8721-05ddbe0cb1e9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.685981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 201/404] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:46 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/5fc4d28cded84afba73b053d18d4aa23/roles/5a5e7aced6d84267bd9e1fa0c1016420 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:46.686814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cae9c-8c68-418e-883e-1fac6df8651a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.715884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 204/405] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:46 2022] POST /identity/v3/users => generated 247 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:46.717339 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62192-1db8-4795-8721-05ddbe0cb1e9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.843666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cae9c-8c68-418e-883e-1fac6df8651a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.852128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62192-1db8-4795-8721-05ddbe0cb1e9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:46.893182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:46.894700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bc55c-95cc-4af8-9ff5-9a23792d80fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.895395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bc55c-95cc-4af8-9ff5-9a23792d80fc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.895740 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bc55c-95cc-4af8-9ff5-9a23792d80fc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.896090 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bc55c-95cc-4af8-9ff5-9a23792d80fc None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.896902 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bc55c-95cc-4af8-9ff5-9a23792d80fc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:46.897308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bc55c-95cc-4af8-9ff5-9a23792d80fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.898255 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852389a1-23d0-4d82-ba43-f8443c4f275c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:46.898876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852389a1-23d0-4d82-ba43-f8443c4f275c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:46.899078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bc55c-95cc-4af8-9ff5-9a23792d80fc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.899179 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852389a1-23d0-4d82-ba43-f8443c4f275c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:46.899421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852389a1-23d0-4d82-ba43-f8443c4f275c None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:46.900250 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852389a1-23d0-4d82-ba43-f8443c4f275c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:46.900566 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852389a1-23d0-4d82-ba43-f8443c4f275c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:46.902946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852389a1-23d0-4d82-ba43-f8443c4f275c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:46.921297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 202/406] 10.222.0.30 () {56 vars in 1139 bytes} [Sat Oct 15 12:43:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:46.922320 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bc55c-95cc-4af8-9ff5-9a23792d80fc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.926350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 205/407] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:46.931252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852389a1-23d0-4d82-ba43-f8443c4f275c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:46.972173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:47.056311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852389a1-23d0-4d82-ba43-f8443c4f275c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:47.072095 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bc55c-95cc-4af8-9ff5-9a23792d80fc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:47.094727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce26a45-6e00-4dde-821d-c9df22f651a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:47.095442 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce26a45-6e00-4dde-821d-c9df22f651a0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:47.095817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce26a45-6e00-4dde-821d-c9df22f651a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:47.096188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce26a45-6e00-4dde-821d-c9df22f651a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:47.107887 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce26a45-6e00-4dde-821d-c9df22f651a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:47.109651 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce26a45-6e00-4dde-821d-c9df22f651a0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:47.119584 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7f24b-9029-479f-8515-7cbde9cdafc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:47.120488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7f24b-9029-479f-8515-7cbde9cdafc2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:47.120939 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7f24b-9029-479f-8515-7cbde9cdafc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:47.121383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7f24b-9029-479f-8515-7cbde9cdafc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0ce26a45-6e00-4dde-821d-c9df22f651a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:47.124737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:47.125856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:47.126910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:47.126910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:47.126910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:47.128325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 206/408] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:47.129258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce26a45-6e00-4dde-821d-c9df22f651a0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:47.137566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7f24b-9029-479f-8515-7cbde9cdafc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:47.139843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7f24b-9029-479f-8515-7cbde9cdafc2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-51e7f24b-9029-479f-8515-7cbde9cdafc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:47.161949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:47.163089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:47.164132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:47.164132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:47.164132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:47.171886 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 203/409] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:47.171886 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7f24b-9029-479f-8515-7cbde9cdafc2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:47.182403 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:47.244175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce26a45-6e00-4dde-821d-c9df22f651a0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:47.287701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033c88d-c9e9-4b78-9558-16f746cd7c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:47.288649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033c88d-c9e9-4b78-9558-16f746cd7c71 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:47.289388 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033c88d-c9e9-4b78-9558-16f746cd7c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:47.289875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033c88d-c9e9-4b78-9558-16f746cd7c71 None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/44b2b449acaa45f099c8a56ecbf20ca6/roles/5a5e7aced6d84267bd9e1fa0c1016420` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:47.294666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:47.307217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7f24b-9029-479f-8515-7cbde9cdafc2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:47.312597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033c88d-c9e9-4b78-9558-16f746cd7c71 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=44b2b449acaa45f099c8a56ecbf20ca6, role_id=5a5e7aced6d84267bd9e1fa0c1016420)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:47.314482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033c88d-c9e9-4b78-9558-16f746cd7c71 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:47.344608 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b7fc7-6aad-428a-a53e-48798c6e0b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:47.345477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b7fc7-6aad-428a-a53e-48798c6e0b33 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:47.345814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b7fc7-6aad-428a-a53e-48798c6e0b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:47.346181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b7fc7-6aad-428a-a53e-48798c6e0b33 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:47.346946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b7fc7-6aad-428a-a53e-48798c6e0b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:47.347360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b7fc7-6aad-428a-a53e-48798c6e0b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:47.348821 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b7fc7-6aad-428a-a53e-48798c6e0b33 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:47.360776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 204/410] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:47.365231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b7fc7-6aad-428a-a53e-48798c6e0b33 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:47.398088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 207/411] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:47 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/44b2b449acaa45f099c8a56ecbf20ca6/roles/5a5e7aced6d84267bd9e1fa0c1016420 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:47.399264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033c88d-c9e9-4b78-9558-16f746cd7c71 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:47.560351 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:47.566012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b7fc7-6aad-428a-a53e-48798c6e0b33 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:47.571213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033c88d-c9e9-4b78-9558-16f746cd7c71 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:47.612797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469ae20-f6d1-428b-9924-b76393244802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:47.613802 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ae20-f6d1-428b-9924-b76393244802 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:47.614226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ae20-f6d1-428b-9924-b76393244802 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:47.614658 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ae20-f6d1-428b-9924-b76393244802 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:47.624588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469ae20-f6d1-428b-9924-b76393244802 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:47.626313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469ae20-f6d1-428b-9924-b76393244802 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f469ae20-f6d1-428b-9924-b76393244802 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:47.639104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:47.640199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:47.641270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:47.641270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:47.641270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:47.641270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:47.641270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:47.641270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:47.641270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:47.641270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:47.643459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 208/412] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:47.649361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469ae20-f6d1-428b-9924-b76393244802 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:47.654009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af38d-67dd-45a9-ac6d-38adfa2dd9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:47.655194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af38d-67dd-45a9-ac6d-38adfa2dd9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:47.655194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af38d-67dd-45a9-ac6d-38adfa2dd9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:47.655533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af38d-67dd-45a9-ac6d-38adfa2dd9b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:47.656293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9af38d-67dd-45a9-ac6d-38adfa2dd9b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:47.656726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9af38d-67dd-45a9-ac6d-38adfa2dd9b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:47.658357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9af38d-67dd-45a9-ac6d-38adfa2dd9b8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:47.672948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 205/413] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:47.678252 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af38d-67dd-45a9-ac6d-38adfa2dd9b8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:47.694286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:47.769421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469ae20-f6d1-428b-9924-b76393244802 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:47.813776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e56540-afa7-434c-a752-2587f0930503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:47.817323 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e56540-afa7-434c-a752-2587f0930503 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:47.817830 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e56540-afa7-434c-a752-2587f0930503 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:47.818363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e56540-afa7-434c-a752-2587f0930503 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:47.819252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e56540-afa7-434c-a752-2587f0930503 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:47.819979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e56540-afa7-434c-a752-2587f0930503 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:47.822579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e56540-afa7-434c-a752-2587f0930503 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:47.843392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af38d-67dd-45a9-ac6d-38adfa2dd9b8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:47.848601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 209/414] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:47.856940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e56540-afa7-434c-a752-2587f0930503 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:47.883679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:47.922361 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872b583-48bc-470d-a8ea-2d06c2cbf792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:47.923720 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872b583-48bc-470d-a8ea-2d06c2cbf792 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:47.924494 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872b583-48bc-470d-a8ea-2d06c2cbf792 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:47.925889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872b583-48bc-470d-a8ea-2d06c2cbf792 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:47.941174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872b583-48bc-470d-a8ea-2d06c2cbf792 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:47.944499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872b583-48bc-470d-a8ea-2d06c2cbf792 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4872b583-48bc-470d-a8ea-2d06c2cbf792 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:47.973380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:47.974287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:47.975141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:47.975141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:47.975141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:47.975141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:47.975141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:47.975141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:47.975141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:47.975141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:47.978322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 206/415] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:47.984668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872b583-48bc-470d-a8ea-2d06c2cbf792 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:48.067079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e56540-afa7-434c-a752-2587f0930503 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:48.126199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:48.141846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ac279-f0a8-4c37-97a6-1872ced8fb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:48.142847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ac279-f0a8-4c37-97a6-1872ced8fb25 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:48.143330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ac279-f0a8-4c37-97a6-1872ced8fb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:48.143802 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ac279-f0a8-4c37-97a6-1872ced8fb25 None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/7a459d7bb6ea42ed93c5bd8eea8a37b8/roles/5a5e7aced6d84267bd9e1fa0c1016420` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:48.173779 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ac279-f0a8-4c37-97a6-1872ced8fb25 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=7a459d7bb6ea42ed93c5bd8eea8a37b8, role_id=5a5e7aced6d84267bd9e1fa0c1016420)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:48.175348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ac279-f0a8-4c37-97a6-1872ced8fb25 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:48.182850 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872b583-48bc-470d-a8ea-2d06c2cbf792 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:48.220452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885658e-d5d6-4ee0-9863-df425994eef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:48.221505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885658e-d5d6-4ee0-9863-df425994eef0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:48.222004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885658e-d5d6-4ee0-9863-df425994eef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:48.222499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885658e-d5d6-4ee0-9863-df425994eef0 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:48.223473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885658e-d5d6-4ee0-9863-df425994eef0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:48.223988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885658e-d5d6-4ee0-9863-df425994eef0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:48.225660 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885658e-d5d6-4ee0-9863-df425994eef0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:48.238774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 207/416] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:48.247101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885658e-d5d6-4ee0-9863-df425994eef0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:48.252632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 210/417] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:47 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/7a459d7bb6ea42ed93c5bd8eea8a37b8/roles/5a5e7aced6d84267bd9e1fa0c1016420 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:48.386577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885658e-d5d6-4ee0-9863-df425994eef0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:48.416587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:48.424623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02248a-bfda-47aa-9418-88e9b3b35ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:48.425418 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02248a-bfda-47aa-9418-88e9b3b35ca2 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:48.425782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02248a-bfda-47aa-9418-88e9b3b35ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:48.426167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02248a-bfda-47aa-9418-88e9b3b35ca2 None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/264bc0b947614ec2a554d627e9e2da56/roles/5a5e7aced6d84267bd9e1fa0c1016420` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:48.442819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02248a-bfda-47aa-9418-88e9b3b35ca2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=264bc0b947614ec2a554d627e9e2da56, role_id=5a5e7aced6d84267bd9e1fa0c1016420)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:48.444418 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02248a-bfda-47aa-9418-88e9b3b35ca2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:48.509558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 208/418] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:48 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/264bc0b947614ec2a554d627e9e2da56/roles/5a5e7aced6d84267bd9e1fa0c1016420 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:48.954058 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d8669-06b9-4890-b19b-800f3f50741d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:48.954917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d8669-06b9-4890-b19b-800f3f50741d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:48.955327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d8669-06b9-4890-b19b-800f3f50741d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:48.957056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 211/419] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:48.962555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8957a7-daaa-4e65-bbca-310d74458c6f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:48.963274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8957a7-daaa-4e65-bbca-310d74458c6f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:48.963687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8957a7-daaa-4e65-bbca-310d74458c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:48.994201 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5a8957a7-daaa-4e65-bbca-310d74458c6f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:48.996236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5a8957a7-daaa-4e65-bbca-310d74458c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:49.091371 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 209/420] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:49.187070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3d075-570a-4d60-a6d1-f5e6c1ab4676 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:49.187588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3d075-570a-4d60-a6d1-f5e6c1ab4676 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:49.187999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3d075-570a-4d60-a6d1-f5e6c1ab4676 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:49.189782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 212/421] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:49.194767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35787510-7024-4590-ba53-71f1b2f4f7b8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:49.195058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35787510-7024-4590-ba53-71f1b2f4f7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:49.195338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35787510-7024-4590-ba53-71f1b2f4f7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:49.218798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-35787510-7024-4590-ba53-71f1b2f4f7b8 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:49.219934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-35787510-7024-4590-ba53-71f1b2f4f7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:49.311809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 210/422] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:49.371518 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3d075-570a-4d60-a6d1-f5e6c1ab4676 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:49.455589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:49.507402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3d075-570a-4d60-a6d1-f5e6c1ab4676 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:49.562356 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cfcb5-19eb-4a8f-95b3-6ce75658f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:49.563011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cfcb5-19eb-4a8f-95b3-6ce75658f0ec None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:49.563292 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cfcb5-19eb-4a8f-95b3-6ce75658f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:49.563569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cfcb5-19eb-4a8f-95b3-6ce75658f0ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:49.576926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cfcb5-19eb-4a8f-95b3-6ce75658f0ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-763cfcb5-19eb-4a8f-95b3-6ce75658f0ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:49.581944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:49.582819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:49.583633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 213/423] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:49.591296 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35787510-7024-4590-ba53-71f1b2f4f7b8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:49.632319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cfcb5-19eb-4a8f-95b3-6ce75658f0ec None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:49.671127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:49.789587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35787510-7024-4590-ba53-71f1b2f4f7b8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:49.821254 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cfcb5-19eb-4a8f-95b3-6ce75658f0ec None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:49.858011 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a92a4-4ff0-43db-a2ab-4d551c6cc6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:49.858635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a92a4-4ff0-43db-a2ab-4d551c6cc6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:49.858911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a92a4-4ff0-43db-a2ab-4d551c6cc6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:49.859183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a92a4-4ff0-43db-a2ab-4d551c6cc6b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:49.859848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a92a4-4ff0-43db-a2ab-4d551c6cc6b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:49.860149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a92a4-4ff0-43db-a2ab-4d551c6cc6b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:49.861555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a92a4-4ff0-43db-a2ab-4d551c6cc6b7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:49.868567 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:49.882504 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31039608-f7c5-486d-964a-cc9f96d1d05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:49.883142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31039608-f7c5-486d-964a-cc9f96d1d05e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:49.883425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31039608-f7c5-486d-964a-cc9f96d1d05e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:49.883676 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31039608-f7c5-486d-964a-cc9f96d1d05e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:49.885485 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 211/424] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:49.891169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a92a4-4ff0-43db-a2ab-4d551c6cc6b7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:49.894855 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31039608-f7c5-486d-964a-cc9f96d1d05e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-31039608-f7c5-486d-964a-cc9f96d1d05e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:49.901119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:49.902723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:49.903682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:49.906172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 214/425] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:49.915848 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31039608-f7c5-486d-964a-cc9f96d1d05e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:50.045538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:50.051656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a92a4-4ff0-43db-a2ab-4d551c6cc6b7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:50.101441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728567b8-bc99-4778-9967-b1204dcbe5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:50.102444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728567b8-bc99-4778-9967-b1204dcbe5ff None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.102974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728567b8-bc99-4778-9967-b1204dcbe5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.105385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728567b8-bc99-4778-9967-b1204dcbe5ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.125475 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728567b8-bc99-4778-9967-b1204dcbe5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:50.127454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728567b8-bc99-4778-9967-b1204dcbe5ff None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:50.135143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31039608-f7c5-486d-964a-cc9f96d1d05e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-728567b8-bc99-4778-9967-b1204dcbe5ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:50.152873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:50.153858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:50.154727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:50.154727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:50.154727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:50.154727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 212/426] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:50.166192 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728567b8-bc99-4778-9967-b1204dcbe5ff None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:50.180247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340429de-161a-42c7-8b72-fc472f8e0681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:50.180856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340429de-161a-42c7-8b72-fc472f8e0681 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.181338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340429de-161a-42c7-8b72-fc472f8e0681 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.181454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340429de-161a-42c7-8b72-fc472f8e0681 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.182101 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340429de-161a-42c7-8b72-fc472f8e0681 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:50.182408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340429de-161a-42c7-8b72-fc472f8e0681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:50.183941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340429de-161a-42c7-8b72-fc472f8e0681 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:50.193414 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 215/427] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:50.194741 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879a7f2-4bff-4f1d-90f0-10e15d8b573b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.195031 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879a7f2-4bff-4f1d-90f0-10e15d8b573b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.195302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879a7f2-4bff-4f1d-90f0-10e15d8b573b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.196634 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 216/428] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:50.199568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879a7f2-4bff-4f1d-90f0-10e15d8b573b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:50.258076 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:50.314404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728567b8-bc99-4778-9967-b1204dcbe5ff None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:50.320729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879a7f2-4bff-4f1d-90f0-10e15d8b573b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:50.362327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781a864-d6db-4a44-8cea-c02207c2b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:50.363103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781a864-d6db-4a44-8cea-c02207c2b174 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.363419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62778a-3fb5-433e-b881-e5f937b14e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:50.363535 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781a864-d6db-4a44-8cea-c02207c2b174 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.363796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781a864-d6db-4a44-8cea-c02207c2b174 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.364511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781a864-d6db-4a44-8cea-c02207c2b174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:50.364688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62778a-3fb5-433e-b881-e5f937b14e68 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.364958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781a864-d6db-4a44-8cea-c02207c2b174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:50.365342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62778a-3fb5-433e-b881-e5f937b14e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.365784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62778a-3fb5-433e-b881-e5f937b14e68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.366728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781a864-d6db-4a44-8cea-c02207c2b174 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:50.378236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62778a-3fb5-433e-b881-e5f937b14e68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:50.379914 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 213/429] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:50.380148 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62778a-3fb5-433e-b881-e5f937b14e68 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:50.381678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e23d95-5869-4224-8f8e-05bfc137978c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.382037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e23d95-5869-4224-8f8e-05bfc137978c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.382389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e23d95-5869-4224-8f8e-05bfc137978c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ee62778a-3fb5-433e-b881-e5f937b14e68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:50.396060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:50.397014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:50.397874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:50.397874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:50.397874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:50.399923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 217/430] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:50.400697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-00e23d95-5869-4224-8f8e-05bfc137978c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:50.401056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62778a-3fb5-433e-b881-e5f937b14e68 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:50.402730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-00e23d95-5869-4224-8f8e-05bfc137978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:50.418063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:50.474900 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 214/431] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:50.476026 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e23d95-5869-4224-8f8e-05bfc137978c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:50.503256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:50.510274 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62778a-3fb5-433e-b881-e5f937b14e68 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:50.561482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3302f-ad9d-4315-ba70-e750c24c213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:50.562396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3302f-ad9d-4315-ba70-e750c24c213c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.562873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3302f-ad9d-4315-ba70-e750c24c213c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.563342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3302f-ad9d-4315-ba70-e750c24c213c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.571869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3302f-ad9d-4315-ba70-e750c24c213c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:50.573690 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3302f-ad9d-4315-ba70-e750c24c213c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7de3302f-ad9d-4315-ba70-e750c24c213c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:50.586938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:50.587878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 15 12:43:50.588765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:50.591527 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 218/432] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:50.600413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e23d95-5869-4224-8f8e-05bfc137978c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:50.601610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3302f-ad9d-4315-ba70-e750c24c213c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:50.636994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa25bd1-9a6f-4c72-9489-092c8430081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:50.637711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa25bd1-9a6f-4c72-9489-092c8430081e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.637995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa25bd1-9a6f-4c72-9489-092c8430081e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.638272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa25bd1-9a6f-4c72-9489-092c8430081e None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.638923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa25bd1-9a6f-4c72-9489-092c8430081e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:50.639233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa25bd1-9a6f-4c72-9489-092c8430081e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:50.640685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa25bd1-9a6f-4c72-9489-092c8430081e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:50.658322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 215/433] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:50.663003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa25bd1-9a6f-4c72-9489-092c8430081e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:50.681571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:50.797294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa25bd1-9a6f-4c72-9489-092c8430081e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:50.835087 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e3fcf-82f0-4600-8c27-50843cabb30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:50.835659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e3fcf-82f0-4600-8c27-50843cabb30c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.835944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e3fcf-82f0-4600-8c27-50843cabb30c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.836238 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e3fcf-82f0-4600-8c27-50843cabb30c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.844775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e3fcf-82f0-4600-8c27-50843cabb30c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:50.846384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e3fcf-82f0-4600-8c27-50843cabb30c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:50.850081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3302f-ad9d-4315-ba70-e750c24c213c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-255e3fcf-82f0-4600-8c27-50843cabb30c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:50.861979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:50.863007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:50.863891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 216/434] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:43:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:50.864444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e3fcf-82f0-4600-8c27-50843cabb30c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:50.903549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22618f1-01b5-4677-b700-da4e74f1892d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:50.904163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22618f1-01b5-4677-b700-da4e74f1892d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:50.904445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22618f1-01b5-4677-b700-da4e74f1892d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:50.904723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22618f1-01b5-4677-b700-da4e74f1892d None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:50.905535 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22618f1-01b5-4677-b700-da4e74f1892d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:50.905908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22618f1-01b5-4677-b700-da4e74f1892d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:50.908467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22618f1-01b5-4677-b700-da4e74f1892d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:50.933623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 219/435] 10.222.0.30 () {56 vars in 1137 bytes} [Sat Oct 15 12:43:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:50.935186 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22618f1-01b5-4677-b700-da4e74f1892d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:50.942227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:50.956273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:50.993532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e3fcf-82f0-4600-8c27-50843cabb30c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:51.044688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314113c5-3cf5-4630-ab25-077fd04a4060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:51.045958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314113c5-3cf5-4630-ab25-077fd04a4060 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.047378 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314113c5-3cf5-4630-ab25-077fd04a4060 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.047810 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314113c5-3cf5-4630-ab25-077fd04a4060 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.065397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314113c5-3cf5-4630-ab25-077fd04a4060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:51.069467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22618f1-01b5-4677-b700-da4e74f1892d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-314113c5-3cf5-4630-ab25-077fd04a4060 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:51.071668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:51.072753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:51.073844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:51.080124 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 217/436] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:51.081218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314113c5-3cf5-4630-ab25-077fd04a4060 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:51.117606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544e079-e7c3-488d-8c4a-ca8923041cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:51.118877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544e079-e7c3-488d-8c4a-ca8923041cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.119652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544e079-e7c3-488d-8c4a-ca8923041cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544e079-e7c3-488d-8c4a-ca8923041cb0 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.121419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544e079-e7c3-488d-8c4a-ca8923041cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:51.121885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544e079-e7c3-488d-8c4a-ca8923041cb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:51.123500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544e079-e7c3-488d-8c4a-ca8923041cb0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:51.143341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 220/437] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 12:43:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:51.144972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544e079-e7c3-488d-8c4a-ca8923041cb0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:51.209483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314113c5-3cf5-4630-ab25-077fd04a4060 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:51.217770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:51.233095 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:51.244349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111d910-7421-4138-8f76-a6e3fb775898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:51.246724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111d910-7421-4138-8f76-a6e3fb775898 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.246724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111d910-7421-4138-8f76-a6e3fb775898 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.246724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111d910-7421-4138-8f76-a6e3fb775898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.247835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544e079-e7c3-488d-8c4a-ca8923041cb0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:51.282261 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5503-017b-42c7-b04f-5f8a1d09ec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:51.282867 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5503-017b-42c7-b04f-5f8a1d09ec02 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.283149 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5503-017b-42c7-b04f-5f8a1d09ec02 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.283425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5503-017b-42c7-b04f-5f8a1d09ec02 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.284085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc5503-017b-42c7-b04f-5f8a1d09ec02 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:51.284644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc5503-017b-42c7-b04f-5f8a1d09ec02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:51.286048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc5503-017b-42c7-b04f-5f8a1d09ec02 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:51.294173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 221/438] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:51.294973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5503-017b-42c7-b04f-5f8a1d09ec02 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:51.315625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111d910-7421-4138-8f76-a6e3fb775898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:51.317208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111d910-7421-4138-8f76-a6e3fb775898 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:51.395488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5503-017b-42c7-b04f-5f8a1d09ec02 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:51.415644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 218/439] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:51.416721 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b78ab-84cc-41b3-9ff4-7b8d3b593f61 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.417011 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b78ab-84cc-41b3-9ff4-7b8d3b593f61 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.417340 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b78ab-84cc-41b3-9ff4-7b8d3b593f61 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.418586 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 219/440] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:51.419610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5eaba-a6b8-4e0f-b651-d0ff38a06bd2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.419882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5eaba-a6b8-4e0f-b651-d0ff38a06bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.420161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5eaba-a6b8-4e0f-b651-d0ff38a06bd2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.421365 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 220/441] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:51.422066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5eaba-a6b8-4e0f-b651-d0ff38a06bd2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:51.438958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2dafd-ad5f-4359-b2ea-65b2465fc591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:51.439596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2dafd-ad5f-4359-b2ea-65b2465fc591 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.439880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2dafd-ad5f-4359-b2ea-65b2465fc591 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.440161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2dafd-ad5f-4359-b2ea-65b2465fc591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.442125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:51.446149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:51.451715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2dafd-ad5f-4359-b2ea-65b2465fc591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:51.453175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2dafd-ad5f-4359-b2ea-65b2465fc591 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-86e2dafd-ad5f-4359-b2ea-65b2465fc591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:51.467213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:51.468325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:51.469180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:51.469180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:51.469180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:51.469180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 222/442] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:51.469180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2dafd-ad5f-4359-b2ea-65b2465fc591 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:51.523937 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5eaba-a6b8-4e0f-b651-d0ff38a06bd2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:51.559647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74b5a7-37db-4109-a014-38b92dee7604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:51.560278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74b5a7-37db-4109-a014-38b92dee7604 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.560558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74b5a7-37db-4109-a014-38b92dee7604 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.560828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74b5a7-37db-4109-a014-38b92dee7604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.570497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74b5a7-37db-4109-a014-38b92dee7604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:51.571052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2dafd-ad5f-4359-b2ea-65b2465fc591 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:51.571944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74b5a7-37db-4109-a014-38b92dee7604 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-eb74b5a7-37db-4109-a014-38b92dee7604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:51.587227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:51.588147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:51.589064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:51.589064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:51.589064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:51.589064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 221/443] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:51.590392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e35f-56b1-4117-a8b4-a7e1f7e61ac3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.590392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e35f-56b1-4117-a8b4-a7e1f7e61ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.590670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e35f-56b1-4117-a8b4-a7e1f7e61ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.607637 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6c70e35f-56b1-4117-a8b4-a7e1f7e61ac3 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:51.608658 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b24c4e-267a-44bf-8d2f-5520ce3409a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:51.608827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6c70e35f-56b1-4117-a8b4-a7e1f7e61ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:51.609347 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24c4e-267a-44bf-8d2f-5520ce3409a4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.609633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24c4e-267a-44bf-8d2f-5520ce3409a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.609936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24c4e-267a-44bf-8d2f-5520ce3409a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.621901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b24c4e-267a-44bf-8d2f-5520ce3409a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:51.623358 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b24c4e-267a-44bf-8d2f-5520ce3409a4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d2b24c4e-267a-44bf-8d2f-5520ce3409a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:51.637929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:51.638820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:51.639657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:51.639657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:51.639657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:51.639657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 223/444] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:51.640363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061d932-09be-47fd-aee6-833a315c6212 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.640553 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061d932-09be-47fd-aee6-833a315c6212 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.640835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061d932-09be-47fd-aee6-833a315c6212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.659574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2061d932-09be-47fd-aee6-833a315c6212 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:51.660690 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2061d932-09be-47fd-aee6-833a315c6212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:51.676761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 222/445] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:51.677649 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e35f-56b1-4117-a8b4-a7e1f7e61ac3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:51.681843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:51.691147 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:51.729276 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 224/446] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:51.733232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061d932-09be-47fd-aee6-833a315c6212 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:51.797923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e35f-56b1-4117-a8b4-a7e1f7e61ac3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:51.833989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986fde7-8375-4f8e-a210-fc3e48ae33f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:51.834877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986fde7-8375-4f8e-a210-fc3e48ae33f6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.835397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986fde7-8375-4f8e-a210-fc3e48ae33f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.835974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986fde7-8375-4f8e-a210-fc3e48ae33f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.836812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986fde7-8375-4f8e-a210-fc3e48ae33f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:51.837352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986fde7-8375-4f8e-a210-fc3e48ae33f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:51.839251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986fde7-8375-4f8e-a210-fc3e48ae33f6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:51.852910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 223/447] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:51.854083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986fde7-8375-4f8e-a210-fc3e48ae33f6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:51.858828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061d932-09be-47fd-aee6-833a315c6212 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:51.896213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c7fc1-99a9-44b5-b047-b833d57b2f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:51.896822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c7fc1-99a9-44b5-b047-b833d57b2f81 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:51.897096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c7fc1-99a9-44b5-b047-b833d57b2f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:51.897401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c7fc1-99a9-44b5-b047-b833d57b2f81 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:51.898099 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c7fc1-99a9-44b5-b047-b833d57b2f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:51.898407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c7fc1-99a9-44b5-b047-b833d57b2f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:51.899777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c7fc1-99a9-44b5-b047-b833d57b2f81 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:51.912381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 225/448] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:51.912930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c7fc1-99a9-44b5-b047-b833d57b2f81 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:51.925096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:51.945226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:51.971042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986fde7-8375-4f8e-a210-fc3e48ae33f6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee5524-cbb4-475f-8362-5f57136edf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.013509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee5524-cbb4-475f-8362-5f57136edf78 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.013789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee5524-cbb4-475f-8362-5f57136edf78 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.014068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee5524-cbb4-475f-8362-5f57136edf78 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.014708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee5524-cbb4-475f-8362-5f57136edf78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:52.015016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee5524-cbb4-475f-8362-5f57136edf78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.016399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee5524-cbb4-475f-8362-5f57136edf78 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.021681 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c7fc1-99a9-44b5-b047-b833d57b2f81 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.028178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 224/449] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:52.029049 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee5524-cbb4-475f-8362-5f57136edf78 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.055299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345f5bf-7eb3-43c4-970d-d0bd84ae31fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.055930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345f5bf-7eb3-43c4-970d-d0bd84ae31fe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.056211 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345f5bf-7eb3-43c4-970d-d0bd84ae31fe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.056486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345f5bf-7eb3-43c4-970d-d0bd84ae31fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.063666 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345f5bf-7eb3-43c4-970d-d0bd84ae31fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.065124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345f5bf-7eb3-43c4-970d-d0bd84ae31fe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f345f5bf-7eb3-43c4-970d-d0bd84ae31fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:52.077327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:52.078172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:52.079014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:52.079014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:52.079014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:52.079014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:52.079014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:52.079014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:52.079014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:52.079014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:52.079014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 226/450] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:52.079468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345f5bf-7eb3-43c4-970d-d0bd84ae31fe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.129816 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee5524-cbb4-475f-8362-5f57136edf78 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.152895 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:52.163992 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f680-0f1f-470f-9f74-5efa21a68977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.164567 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f680-0f1f-470f-9f74-5efa21a68977 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.164847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f680-0f1f-470f-9f74-5efa21a68977 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.165120 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f680-0f1f-470f-9f74-5efa21a68977 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.165876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:52.172415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9f680-0f1f-470f-9f74-5efa21a68977 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.173903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9f680-0f1f-470f-9f74-5efa21a68977 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.177583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345f5bf-7eb3-43c4-970d-d0bd84ae31fe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-31d9f680-0f1f-470f-9f74-5efa21a68977 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:52.185765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:52.186652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 225/451] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:52.192272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f680-0f1f-470f-9f74-5efa21a68977 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.211864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6c443-745b-4e17-a15c-2c1ef549b0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.212487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6c443-745b-4e17-a15c-2c1ef549b0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.212770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6c443-745b-4e17-a15c-2c1ef549b0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.213040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6c443-745b-4e17-a15c-2c1ef549b0e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.223509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6c443-745b-4e17-a15c-2c1ef549b0e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a9e6c443-745b-4e17-a15c-2c1ef549b0e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:52.228644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:52.229630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:52.231218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 227/452] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:52.231871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6c443-745b-4e17-a15c-2c1ef549b0e9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.287598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f680-0f1f-470f-9f74-5efa21a68977 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.329583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6c443-745b-4e17-a15c-2c1ef549b0e9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.361946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047fa40-93bc-4a74-b347-bbad263033f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.362120 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbaf14-2428-46c8-8c88-89600b46990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.362576 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047fa40-93bc-4a74-b347-bbad263033f2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.362771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbaf14-2428-46c8-8c88-89600b46990a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.362857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047fa40-93bc-4a74-b347-bbad263033f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.362990 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbaf14-2428-46c8-8c88-89600b46990a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.363116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047fa40-93bc-4a74-b347-bbad263033f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.363266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbaf14-2428-46c8-8c88-89600b46990a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.369652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047fa40-93bc-4a74-b347-bbad263033f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.371813 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047fa40-93bc-4a74-b347-bbad263033f2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.374099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbaf14-2428-46c8-8c88-89600b46990a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-abfbaf14-2428-46c8-8c88-89600b46990a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:52.379421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:52.380295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 226/453] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:52.381873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbaf14-2428-46c8-8c88-89600b46990a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-2047fa40-93bc-4a74-b347-bbad263033f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:52.382999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:52.383809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 228/454] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:52.385215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047fa40-93bc-4a74-b347-bbad263033f2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.396102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:52.428594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:52.485172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbaf14-2428-46c8-8c88-89600b46990a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.485366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047fa40-93bc-4a74-b347-bbad263033f2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.518782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0233-8a72-4bbc-a960-c8c36404c0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.519040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beeab0b-90d0-4655-898b-ed1d9404632c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.519389 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0233-8a72-4bbc-a960-c8c36404c0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.519598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beeab0b-90d0-4655-898b-ed1d9404632c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.519696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0233-8a72-4bbc-a960-c8c36404c0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.519881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beeab0b-90d0-4655-898b-ed1d9404632c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.519976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0233-8a72-4bbc-a960-c8c36404c0d8 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.520159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beeab0b-90d0-4655-898b-ed1d9404632c None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.520647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0233-8a72-4bbc-a960-c8c36404c0d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:52.520842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beeab0b-90d0-4655-898b-ed1d9404632c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:52.520996 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0233-8a72-4bbc-a960-c8c36404c0d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.521145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beeab0b-90d0-4655-898b-ed1d9404632c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.522434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0233-8a72-4bbc-a960-c8c36404c0d8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.522606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beeab0b-90d0-4655-898b-ed1d9404632c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.539469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 227/455] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:52.539469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beeab0b-90d0-4655-898b-ed1d9404632c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.546397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 229/456] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:52.546397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0233-8a72-4bbc-a960-c8c36404c0d8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.610566 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:52.643068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beeab0b-90d0-4655-898b-ed1d9404632c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.643266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0233-8a72-4bbc-a960-c8c36404c0d8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.645412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:52.677953 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531c0d0-caed-4710-acc9-e4cd9a117855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.678564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531c0d0-caed-4710-acc9-e4cd9a117855 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.678891 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531c0d0-caed-4710-acc9-e4cd9a117855 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.679097 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2a79f-4583-4b7d-acdc-c0cd3d2dc49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.679201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531c0d0-caed-4710-acc9-e4cd9a117855 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.679706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2a79f-4583-4b7d-acdc-c0cd3d2dc49d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.679795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531c0d0-caed-4710-acc9-e4cd9a117855 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:52.679989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2a79f-4583-4b7d-acdc-c0cd3d2dc49d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.680101 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531c0d0-caed-4710-acc9-e4cd9a117855 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.680266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2a79f-4583-4b7d-acdc-c0cd3d2dc49d None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.680925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2a79f-4583-4b7d-acdc-c0cd3d2dc49d None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:52.681260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2a79f-4583-4b7d-acdc-c0cd3d2dc49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.681563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531c0d0-caed-4710-acc9-e4cd9a117855 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.682680 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2a79f-4583-4b7d-acdc-c0cd3d2dc49d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.693614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 228/457] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:52.693774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 230/458] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:52.699194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2a79f-4583-4b7d-acdc-c0cd3d2dc49d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.699325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531c0d0-caed-4710-acc9-e4cd9a117855 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.798244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531c0d0-caed-4710-acc9-e4cd9a117855 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.799906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2a79f-4583-4b7d-acdc-c0cd3d2dc49d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.823010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:52.831961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157c9c5-385c-4b28-a9f9-a6e8b1601438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.832617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157c9c5-385c-4b28-a9f9-a6e8b1601438 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.832898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157c9c5-385c-4b28-a9f9-a6e8b1601438 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.833176 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157c9c5-385c-4b28-a9f9-a6e8b1601438 None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/b849f213606a4686bfb140fec34d512f/roles/5a5e7aced6d84267bd9e1fa0c1016420` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.833686 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60843d45-7257-4402-871e-0c37be72fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.836966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60843d45-7257-4402-871e-0c37be72fb74 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.836966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60843d45-7257-4402-871e-0c37be72fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.836966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60843d45-7257-4402-871e-0c37be72fb74 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:52.836966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60843d45-7257-4402-871e-0c37be72fb74 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:52.836966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60843d45-7257-4402-871e-0c37be72fb74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.837376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60843d45-7257-4402-871e-0c37be72fb74 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.845369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 229/459] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:52.846158 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60843d45-7257-4402-871e-0c37be72fb74 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.849484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157c9c5-385c-4b28-a9f9-a6e8b1601438 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=b849f213606a4686bfb140fec34d512f, role_id=5a5e7aced6d84267bd9e1fa0c1016420)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:52.850972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157c9c5-385c-4b28-a9f9-a6e8b1601438 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:52.875506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:52.907759 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 231/460] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:43:52 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/b849f213606a4686bfb140fec34d512f/roles/5a5e7aced6d84267bd9e1fa0c1016420 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:43:52.913215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157c9c5-385c-4b28-a9f9-a6e8b1601438 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:52.950418 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60843d45-7257-4402-871e-0c37be72fb74 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:52.988003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451ac18-94ed-4cec-a9bf-798a235a77f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:52.988632 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451ac18-94ed-4cec-a9bf-798a235a77f3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:52.988913 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451ac18-94ed-4cec-a9bf-798a235a77f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:52.989225 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451ac18-94ed-4cec-a9bf-798a235a77f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.035394 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157c9c5-385c-4b28-a9f9-a6e8b1601438 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:53.067249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451ac18-94ed-4cec-a9bf-798a235a77f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:53.068746 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451ac18-94ed-4cec-a9bf-798a235a77f3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:53.076622 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934e628-9082-4ca6-90fb-6091f8922448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:53.077275 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934e628-9082-4ca6-90fb-6091f8922448 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:53.077563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934e628-9082-4ca6-90fb-6091f8922448 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:53.078601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934e628-9082-4ca6-90fb-6091f8922448 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.078601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934e628-9082-4ca6-90fb-6091f8922448 None admin] RBAC: Adding query filter params (role.id=5a5e7aced6d84267bd9e1fa0c1016420, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=5fc4d28cded84afba73b053d18d4aa23) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:53.078876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934e628-9082-4ca6-90fb-6091f8922448 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:53.080327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934e628-9082-4ca6-90fb-6091f8922448 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:53.088149 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 232/461] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/role_assignments?user.id=5fc4d28cded84afba73b053d18d4aa23&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=5a5e7aced6d84267bd9e1fa0c1016420 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:53.089070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934e628-9082-4ca6-90fb-6091f8922448 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:53.106037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:53.151354 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:53.207205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 230/462] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:53.209052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451ac18-94ed-4cec-a9bf-798a235a77f3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:53.230736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934e628-9082-4ca6-90fb-6091f8922448 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:53.274645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1662ac-618c-4de7-922b-6cd6ea007164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:53.275716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1662ac-618c-4de7-922b-6cd6ea007164 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:53.276205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1662ac-618c-4de7-922b-6cd6ea007164 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:53.276484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1662ac-618c-4de7-922b-6cd6ea007164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.289488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1662ac-618c-4de7-922b-6cd6ea007164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:53.290946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1662ac-618c-4de7-922b-6cd6ea007164 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7c1662ac-618c-4de7-922b-6cd6ea007164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:53.308413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:53.309468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:53.310379 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:53.310379 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:53.310379 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:53.310379 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 233/463] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:53.310801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1662ac-618c-4de7-922b-6cd6ea007164 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:53.329984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451ac18-94ed-4cec-a9bf-798a235a77f3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:53.362224 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:53.372552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a951e0-5bd2-4703-8aea-405a749e5006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:53.373913 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a951e0-5bd2-4703-8aea-405a749e5006 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:53.374201 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a951e0-5bd2-4703-8aea-405a749e5006 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:53.374481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a951e0-5bd2-4703-8aea-405a749e5006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.386016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a951e0-5bd2-4703-8aea-405a749e5006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:53.388320 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a951e0-5bd2-4703-8aea-405a749e5006 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-29a951e0-5bd2-4703-8aea-405a749e5006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:53.405690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:53.406796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:53.407644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:53.407644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:53.407644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:53.407644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 231/464] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:53.407644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a951e0-5bd2-4703-8aea-405a749e5006 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:53.445574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:53.448170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1662ac-618c-4de7-922b-6cd6ea007164 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:53.496053 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb5455-505e-466f-812d-fdf9fbdb5b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:53.498189 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb5455-505e-466f-812d-fdf9fbdb5b1c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:53.498612 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb5455-505e-466f-812d-fdf9fbdb5b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:53.499074 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb5455-505e-466f-812d-fdf9fbdb5b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.512547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb5455-505e-466f-812d-fdf9fbdb5b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:53.514353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb5455-505e-466f-812d-fdf9fbdb5b1c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-54bb5455-505e-466f-812d-fdf9fbdb5b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:53.532675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:53.533691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:53.534866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:53.534866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:53.534866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:53.536679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 234/465] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:53.541470 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a951e0-5bd2-4703-8aea-405a749e5006 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:53.542285 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb5455-505e-466f-812d-fdf9fbdb5b1c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:53.585706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30121d3e-c328-41b1-b673-a1cb81b34ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:53.586968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30121d3e-c328-41b1-b673-a1cb81b34ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:53.587464 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30121d3e-c328-41b1-b673-a1cb81b34ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:53.587927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30121d3e-c328-41b1-b673-a1cb81b34ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.589555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30121d3e-c328-41b1-b673-a1cb81b34ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:53.590946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30121d3e-c328-41b1-b673-a1cb81b34ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:53.592554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30121d3e-c328-41b1-b673-a1cb81b34ea6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:53.609792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 232/466] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:53.611469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30121d3e-c328-41b1-b673-a1cb81b34ea6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:53.667284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:53.673269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb5455-505e-466f-812d-fdf9fbdb5b1c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:53.718142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032ad94-4215-422e-ac71-ddf4cfee034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:53.718727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032ad94-4215-422e-ac71-ddf4cfee034a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:53.719014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032ad94-4215-422e-ac71-ddf4cfee034a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:53.719290 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032ad94-4215-422e-ac71-ddf4cfee034a None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.719938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032ad94-4215-422e-ac71-ddf4cfee034a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:53.720244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032ad94-4215-422e-ac71-ddf4cfee034a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:53.721646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032ad94-4215-422e-ac71-ddf4cfee034a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:53.735214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 235/467] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:53.736044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032ad94-4215-422e-ac71-ddf4cfee034a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:53.745123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:53.754105 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30121d3e-c328-41b1-b673-a1cb81b34ea6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:53.799430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbe946-0ed1-4614-86b3-b1456853e4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:53.800154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbe946-0ed1-4614-86b3-b1456853e4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:53.800499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbe946-0ed1-4614-86b3-b1456853e4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:53.800868 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbe946-0ed1-4614-86b3-b1456853e4c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.801608 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbe946-0ed1-4614-86b3-b1456853e4c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:53.801978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbe946-0ed1-4614-86b3-b1456853e4c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:53.804235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbe946-0ed1-4614-86b3-b1456853e4c3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:53.825234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 233/468] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:53.825234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbe946-0ed1-4614-86b3-b1456853e4c3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:53.864174 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032ad94-4215-422e-ac71-ddf4cfee034a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:53.903458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26d80c-f683-4a3e-a2d3-2ed65719518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:53.904521 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26d80c-f683-4a3e-a2d3-2ed65719518b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:53.904985 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26d80c-f683-4a3e-a2d3-2ed65719518b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:53.905479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26d80c-f683-4a3e-a2d3-2ed65719518b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.914144 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26d80c-f683-4a3e-a2d3-2ed65719518b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:53.915878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26d80c-f683-4a3e-a2d3-2ed65719518b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1e26d80c-f683-4a3e-a2d3-2ed65719518b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:53.929969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:53.930900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 12:43:53.931815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:53.935030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 236/469] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:53.936106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26d80c-f683-4a3e-a2d3-2ed65719518b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:53.944821 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:53.949746 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbe946-0ed1-4614-86b3-b1456853e4c3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:53.988891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d73f76-5468-428c-9409-464d3cabe00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:53.989901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d73f76-5468-428c-9409-464d3cabe00e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:53.990361 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d73f76-5468-428c-9409-464d3cabe00e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:53.990823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d73f76-5468-428c-9409-464d3cabe00e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:53.999830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d73f76-5468-428c-9409-464d3cabe00e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:54.001671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d73f76-5468-428c-9409-464d3cabe00e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-52d73f76-5468-428c-9409-464d3cabe00e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:54.015887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:54.016777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 12:43:54.017735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:54.019255 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:54.025386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 234/470] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:54.026496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d73f76-5468-428c-9409-464d3cabe00e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:54.068251 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26d80c-f683-4a3e-a2d3-2ed65719518b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:54.112380 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044151ca-b642-4520-b3cb-e7d88b81554d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:54.113299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044151ca-b642-4520-b3cb-e7d88b81554d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:54.113833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044151ca-b642-4520-b3cb-e7d88b81554d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:54.116191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044151ca-b642-4520-b3cb-e7d88b81554d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:54.128655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044151ca-b642-4520-b3cb-e7d88b81554d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:54.130614 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044151ca-b642-4520-b3cb-e7d88b81554d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-044151ca-b642-4520-b3cb-e7d88b81554d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:54.147734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:54.148749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:54.149617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:54.149617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:54.149617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:54.149617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:54.149617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:54.149617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:54.149617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:54.149617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:54.150646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d73f76-5468-428c-9409-464d3cabe00e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:54.153266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 237/471] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:54.154315 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044151ca-b642-4520-b3cb-e7d88b81554d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:54.201026 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508287ef-236b-4ea9-89ef-c8c8bdae8de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:54.201665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508287ef-236b-4ea9-89ef-c8c8bdae8de8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:54.202352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508287ef-236b-4ea9-89ef-c8c8bdae8de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:54.202633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508287ef-236b-4ea9-89ef-c8c8bdae8de8 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:54.203642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508287ef-236b-4ea9-89ef-c8c8bdae8de8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:54.204046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508287ef-236b-4ea9-89ef-c8c8bdae8de8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:54.206282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508287ef-236b-4ea9-89ef-c8c8bdae8de8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:54.232855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 235/472] 10.222.0.30 () {56 vars in 1137 bytes} [Sat Oct 15 12:43:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:54.233834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508287ef-236b-4ea9-89ef-c8c8bdae8de8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:54.259150 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:54.263568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044151ca-b642-4520-b3cb-e7d88b81554d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:54.290591 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:54.302908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611829e4-1099-4e0a-bcff-a385fd4c5937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:54.303623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611829e4-1099-4e0a-bcff-a385fd4c5937 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:54.303910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611829e4-1099-4e0a-bcff-a385fd4c5937 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:54.304308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611829e4-1099-4e0a-bcff-a385fd4c5937 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:54.305273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611829e4-1099-4e0a-bcff-a385fd4c5937 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:54.305597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611829e4-1099-4e0a-bcff-a385fd4c5937 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:54.307095 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611829e4-1099-4e0a-bcff-a385fd4c5937 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:54.325952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 238/473] 10.222.0.30 () {56 vars in 1139 bytes} [Sat Oct 15 12:43:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:54.326681 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611829e4-1099-4e0a-bcff-a385fd4c5937 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:54.365878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508287ef-236b-4ea9-89ef-c8c8bdae8de8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:54.402444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac6104-8692-4db6-b151-845a8aaea2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:54.403065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6104-8692-4db6-b151-845a8aaea2bb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:54.403347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6104-8692-4db6-b151-845a8aaea2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:54.403623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6104-8692-4db6-b151-845a8aaea2bb None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:54.404322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac6104-8692-4db6-b151-845a8aaea2bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:54.404762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac6104-8692-4db6-b151-845a8aaea2bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:54.406310 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac6104-8692-4db6-b151-845a8aaea2bb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:54.425385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 236/474] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:54.426467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac6104-8692-4db6-b151-845a8aaea2bb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:54.450742 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611829e4-1099-4e0a-bcff-a385fd4c5937 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:54.490277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c06216-23a5-4fbf-acba-0dc7a66d590c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:54.490830 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c06216-23a5-4fbf-acba-0dc7a66d590c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:54.491114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c06216-23a5-4fbf-acba-0dc7a66d590c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:54.491433 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c06216-23a5-4fbf-acba-0dc7a66d590c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:54.507573 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:54.530857 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac6104-8692-4db6-b151-845a8aaea2bb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:54.547648 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:54.569359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c06216-23a5-4fbf-acba-0dc7a66d590c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:54.570216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a33d7-b16d-43fa-bddd-b98e87ea0b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:54.570808 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c06216-23a5-4fbf-acba-0dc7a66d590c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:54.571159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a33d7-b16d-43fa-bddd-b98e87ea0b98 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:54.571574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a33d7-b16d-43fa-bddd-b98e87ea0b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:54.571984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a33d7-b16d-43fa-bddd-b98e87ea0b98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:54.641863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a33d7-b16d-43fa-bddd-b98e87ea0b98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:54.643541 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a33d7-b16d-43fa-bddd-b98e87ea0b98 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:54.685210 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 239/475] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:54.686148 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c06216-23a5-4fbf-acba-0dc7a66d590c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:54.722753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:54.748327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 237/476] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 12:43:54.749423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a33d7-b16d-43fa-bddd-b98e87ea0b98 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:54.784820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:54.805670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c06216-23a5-4fbf-acba-0dc7a66d590c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:54.846241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc62b0-6f3d-43f9-b0b3-ee2dbe7dfaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:54.846890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc62b0-6f3d-43f9-b0b3-ee2dbe7dfaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:54.847177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc62b0-6f3d-43f9-b0b3-ee2dbe7dfaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:54.847462 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc62b0-6f3d-43f9-b0b3-ee2dbe7dfaf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:54.848242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc62b0-6f3d-43f9-b0b3-ee2dbe7dfaf9 None admin] RBAC: Adding query filter params (role.id=5a5e7aced6d84267bd9e1fa0c1016420, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=44b2b449acaa45f099c8a56ecbf20ca6) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:54.848563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc62b0-6f3d-43f9-b0b3-ee2dbe7dfaf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:54.850248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc62b0-6f3d-43f9-b0b3-ee2dbe7dfaf9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:54.856467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a33d7-b16d-43fa-bddd-b98e87ea0b98 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:54.862437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 240/477] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:54 2022] GET /identity/v3/role_assignments?user.id=44b2b449acaa45f099c8a56ecbf20ca6&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=5a5e7aced6d84267bd9e1fa0c1016420 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:54.862437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc62b0-6f3d-43f9-b0b3-ee2dbe7dfaf9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:54.947182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071520a-b2c2-44f3-9865-6596193fa1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:54.947849 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071520a-b2c2-44f3-9865-6596193fa1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:54.948135 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071520a-b2c2-44f3-9865-6596193fa1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:54.948446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071520a-b2c2-44f3-9865-6596193fa1e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:54.972506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071520a-b2c2-44f3-9865-6596193fa1e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:54.974772 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071520a-b2c2-44f3-9865-6596193fa1e9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f071520a-b2c2-44f3-9865-6596193fa1e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:55.005223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:55.006319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:55.007165 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:55.007165 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:55.007165 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:55.009145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 238/478] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:55.014746 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071520a-b2c2-44f3-9865-6596193fa1e9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:55.038721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc62b0-6f3d-43f9-b0b3-ee2dbe7dfaf9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:55.073688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:55.078547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff37ba4-6073-48c7-84f7-1f55f40cf51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:55.079463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff37ba4-6073-48c7-84f7-1f55f40cf51b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.079890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff37ba4-6073-48c7-84f7-1f55f40cf51b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.080309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff37ba4-6073-48c7-84f7-1f55f40cf51b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.092598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff37ba4-6073-48c7-84f7-1f55f40cf51b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:55.094431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff37ba4-6073-48c7-84f7-1f55f40cf51b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:55.102097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5ff37ba4-6073-48c7-84f7-1f55f40cf51b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:55.111894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:55.113304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:55.114282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:55.114282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:55.114282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:55.114526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 241/479] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:55.119895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff37ba4-6073-48c7-84f7-1f55f40cf51b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:55.198065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071520a-b2c2-44f3-9865-6596193fa1e9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:55.244377 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff37ba4-6073-48c7-84f7-1f55f40cf51b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:55.250606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1081e-59b6-471d-9c15-a09133104d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:55.251828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1081e-59b6-471d-9c15-a09133104d55 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.252339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1081e-59b6-471d-9c15-a09133104d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.253484 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1081e-59b6-471d-9c15-a09133104d55 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.254351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1081e-59b6-471d-9c15-a09133104d55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:55.255388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1081e-59b6-471d-9c15-a09133104d55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:55.257728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1081e-59b6-471d-9c15-a09133104d55 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:55.272850 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 239/480] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:55.273136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e9b5c-6dca-421f-81e2-84a6f22f084f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.273451 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e9b5c-6dca-421f-81e2-84a6f22f084f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.273726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e9b5c-6dca-421f-81e2-84a6f22f084f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.275178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 240/481] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:55.280953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e9b5c-6dca-421f-81e2-84a6f22f084f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:55.288355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93d69c-fbf7-4d36-aad1-70c15f84eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:55.288976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93d69c-fbf7-4d36-aad1-70c15f84eb63 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.291578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93d69c-fbf7-4d36-aad1-70c15f84eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.291578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93d69c-fbf7-4d36-aad1-70c15f84eb63 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.291578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93d69c-fbf7-4d36-aad1-70c15f84eb63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:55.291578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93d69c-fbf7-4d36-aad1-70c15f84eb63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:55.291979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93d69c-fbf7-4d36-aad1-70c15f84eb63 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:55.306841 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 242/482] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:55.308645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa88f3a-c0ec-46c1-8b78-bc4e355481ce None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.309053 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa88f3a-c0ec-46c1-8b78-bc4e355481ce None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.309364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa88f3a-c0ec-46c1-8b78-bc4e355481ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.328436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8aa88f3a-c0ec-46c1-8b78-bc4e355481ce None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:55.329612 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8aa88f3a-c0ec-46c1-8b78-bc4e355481ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:55.399642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:55.403461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:55.412963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e9b5c-6dca-421f-81e2-84a6f22f084f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:55.425286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 243/483] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:55.426184 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa88f3a-c0ec-46c1-8b78-bc4e355481ce None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:55.465180 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b716e22-2298-441f-acb7-dfa481a1650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:55.465845 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b716e22-2298-441f-acb7-dfa481a1650c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.466123 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b716e22-2298-441f-acb7-dfa481a1650c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.466447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b716e22-2298-441f-acb7-dfa481a1650c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.475015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b716e22-2298-441f-acb7-dfa481a1650c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:55.476461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b716e22-2298-441f-acb7-dfa481a1650c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6b716e22-2298-441f-acb7-dfa481a1650c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:55.489783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:55.490655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:55.491500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:55.491500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:55.491500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:55.491500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:55.491500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:55.491500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:55.491500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:55.491500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:55.491500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 241/484] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:55.497612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b716e22-2298-441f-acb7-dfa481a1650c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:55.602283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa88f3a-c0ec-46c1-8b78-bc4e355481ce None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:55.623082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b716e22-2298-441f-acb7-dfa481a1650c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:55.640472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10905f-2fcc-4c66-884f-6a14a35d19a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:55.641111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10905f-2fcc-4c66-884f-6a14a35d19a4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.641429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10905f-2fcc-4c66-884f-6a14a35d19a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.641702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10905f-2fcc-4c66-884f-6a14a35d19a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.648796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10905f-2fcc-4c66-884f-6a14a35d19a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:55.650878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10905f-2fcc-4c66-884f-6a14a35d19a4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:55.662180 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145eee2-35d1-43ae-8878-8dac96d22f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:55.663064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145eee2-35d1-43ae-8878-8dac96d22f27 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.663565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145eee2-35d1-43ae-8878-8dac96d22f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.664054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145eee2-35d1-43ae-8878-8dac96d22f27 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8b10905f-2fcc-4c66-884f-6a14a35d19a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:55.664524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:55.665361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:55.666220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:55.666220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:55.666220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:55.666220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:55.666220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:55.666220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:55.666220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:55.666220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:55.667885 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145eee2-35d1-43ae-8878-8dac96d22f27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:55.668413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145eee2-35d1-43ae-8878-8dac96d22f27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:55.669524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 244/485] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:55.670338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145eee2-35d1-43ae-8878-8dac96d22f27 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:55.670842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625ba0-8ea4-421c-97f6-87687ff2c51b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.671128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625ba0-8ea4-421c-97f6-87687ff2c51b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.671403 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625ba0-8ea4-421c-97f6-87687ff2c51b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.672655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 245/486] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:55.676815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625ba0-8ea4-421c-97f6-87687ff2c51b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:55.691724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 242/487] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:55.693703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6927e-d3d7-403c-ae38-354d98ea7e9a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.694171 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6927e-d3d7-403c-ae38-354d98ea7e9a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.695113 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6927e-d3d7-403c-ae38-354d98ea7e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.719806 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ead6927e-d3d7-403c-ae38-354d98ea7e9a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:55.721323 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ead6927e-d3d7-403c-ae38-354d98ea7e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:55.724953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:55.735160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:55.790424 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625ba0-8ea4-421c-97f6-87687ff2c51b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:55.805217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 243/488] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:55.805217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6927e-d3d7-403c-ae38-354d98ea7e9a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:55.835741 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6915140-ab8f-443a-84e3-124d8e7f57f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:55.836439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6915140-ab8f-443a-84e3-124d8e7f57f1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.836790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6915140-ab8f-443a-84e3-124d8e7f57f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.837223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6915140-ab8f-443a-84e3-124d8e7f57f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.837985 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6915140-ab8f-443a-84e3-124d8e7f57f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:55.838358 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6915140-ab8f-443a-84e3-124d8e7f57f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:55.840203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6915140-ab8f-443a-84e3-124d8e7f57f1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:55.853312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 246/489] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:55.858291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6915140-ab8f-443a-84e3-124d8e7f57f1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:55.915240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6927e-d3d7-403c-ae38-354d98ea7e9a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:55.954804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a1e0a-4912-4efc-82f7-96ff64e74ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:55.955415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a1e0a-4912-4efc-82f7-96ff64e74ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:55.955842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a1e0a-4912-4efc-82f7-96ff64e74ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:55.956122 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a1e0a-4912-4efc-82f7-96ff64e74ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:55.956869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a1e0a-4912-4efc-82f7-96ff64e74ef0 None admin] RBAC: Adding query filter params (role.id=5a5e7aced6d84267bd9e1fa0c1016420, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=264bc0b947614ec2a554d627e9e2da56) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:55.957160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a1e0a-4912-4efc-82f7-96ff64e74ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:55.958635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a1e0a-4912-4efc-82f7-96ff64e74ef0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:55.965626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 244/490] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:55 2022] GET /identity/v3/role_assignments?user.id=264bc0b947614ec2a554d627e9e2da56&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=5a5e7aced6d84267bd9e1fa0c1016420 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:55.966516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a1e0a-4912-4efc-82f7-96ff64e74ef0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:55.971280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:55.986749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6915140-ab8f-443a-84e3-124d8e7f57f1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:56.001022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:56.034126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d5a29-2aa2-40b2-b757-8497dee2147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:56.035426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d5a29-2aa2-40b2-b757-8497dee2147c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:56.035932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d5a29-2aa2-40b2-b757-8497dee2147c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:56.036459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d5a29-2aa2-40b2-b757-8497dee2147c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:56.050016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d5a29-2aa2-40b2-b757-8497dee2147c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-064d5a29-2aa2-40b2-b757-8497dee2147c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:56.055932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:56.056890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:43:56.058258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:56.060479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 247/491] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:43:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:56.074952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d5a29-2aa2-40b2-b757-8497dee2147c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:56.114854 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a1e0a-4912-4efc-82f7-96ff64e74ef0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:56.169344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871e116-7ec7-48be-8236-f9d9994e23ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:56.169993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871e116-7ec7-48be-8236-f9d9994e23ed None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:56.170341 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871e116-7ec7-48be-8236-f9d9994e23ed None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:56.170629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871e116-7ec7-48be-8236-f9d9994e23ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:56.171370 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871e116-7ec7-48be-8236-f9d9994e23ed None admin] RBAC: Adding query filter params (role.id=5a5e7aced6d84267bd9e1fa0c1016420, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=7a459d7bb6ea42ed93c5bd8eea8a37b8) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:56.171686 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871e116-7ec7-48be-8236-f9d9994e23ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:56.173848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871e116-7ec7-48be-8236-f9d9994e23ed None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:56.182667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 245/492] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:55 2022] GET /identity/v3/role_assignments?user.id=7a459d7bb6ea42ed93c5bd8eea8a37b8&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=5a5e7aced6d84267bd9e1fa0c1016420 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:56.185830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871e116-7ec7-48be-8236-f9d9994e23ed None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:56.254050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d5a29-2aa2-40b2-b757-8497dee2147c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:56.286979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:56.314376 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9f7ea-75bf-4250-8cf2-ba63e0cec589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:56.315034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9f7ea-75bf-4250-8cf2-ba63e0cec589 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:56.315274 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9f7ea-75bf-4250-8cf2-ba63e0cec589 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:56.315547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9f7ea-75bf-4250-8cf2-ba63e0cec589 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:56.316221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9f7ea-75bf-4250-8cf2-ba63e0cec589 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:56.316525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9f7ea-75bf-4250-8cf2-ba63e0cec589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:56.318780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9f7ea-75bf-4250-8cf2-ba63e0cec589 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:56.332553 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 248/493] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:43:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:56.333024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871e116-7ec7-48be-8236-f9d9994e23ed None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:56.340172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9f7ea-75bf-4250-8cf2-ba63e0cec589 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:56.402445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:56.407053 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa43d46-07fb-4d26-b113-ba8200e982a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:56.408001 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa43d46-07fb-4d26-b113-ba8200e982a5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:56.408610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa43d46-07fb-4d26-b113-ba8200e982a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:56.409083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa43d46-07fb-4d26-b113-ba8200e982a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:56.424188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa43d46-07fb-4d26-b113-ba8200e982a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:56.425983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa43d46-07fb-4d26-b113-ba8200e982a5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2fa43d46-07fb-4d26-b113-ba8200e982a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:56.442372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:56.443358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:56.444247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:56.444247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:56.444247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:56.446490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 246/494] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:56.452173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa43d46-07fb-4d26-b113-ba8200e982a5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:56.487122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9f7ea-75bf-4250-8cf2-ba63e0cec589 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:56.528579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696155f-d817-48a3-b26a-51da659c37a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:56.529114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696155f-d817-48a3-b26a-51da659c37a2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:56.529994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696155f-d817-48a3-b26a-51da659c37a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:56.530370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696155f-d817-48a3-b26a-51da659c37a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:56.559722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696155f-d817-48a3-b26a-51da659c37a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:56.561237 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696155f-d817-48a3-b26a-51da659c37a2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d696155f-d817-48a3-b26a-51da659c37a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:56.586153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:56.587309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:56.588774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:56.588774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:56.588774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:56.588774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 249/495] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:56.593397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696155f-d817-48a3-b26a-51da659c37a2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:56.615283 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa43d46-07fb-4d26-b113-ba8200e982a5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:56.652113 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:56.687384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd33a0-f579-499d-a344-a9d6e57f82c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:56.688834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd33a0-f579-499d-a344-a9d6e57f82c3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:56.689368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd33a0-f579-499d-a344-a9d6e57f82c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:56.690095 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd33a0-f579-499d-a344-a9d6e57f82c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:56.691112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd33a0-f579-499d-a344-a9d6e57f82c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:56.691958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd33a0-f579-499d-a344-a9d6e57f82c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:56.694034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd33a0-f579-499d-a344-a9d6e57f82c3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:56.721580 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 247/496] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:56.722649 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd33a0-f579-499d-a344-a9d6e57f82c3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:56.723717 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:56.752499 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696155f-d817-48a3-b26a-51da659c37a2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:56.799662 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff513f0-ef1f-44e5-b39b-f1c1182e7580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:56.800344 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff513f0-ef1f-44e5-b39b-f1c1182e7580 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:56.800625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff513f0-ef1f-44e5-b39b-f1c1182e7580 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:56.800899 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff513f0-ef1f-44e5-b39b-f1c1182e7580 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:56.801571 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff513f0-ef1f-44e5-b39b-f1c1182e7580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:56.801882 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff513f0-ef1f-44e5-b39b-f1c1182e7580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:56.803266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff513f0-ef1f-44e5-b39b-f1c1182e7580 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:56.818133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 250/497] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:56.826525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff513f0-ef1f-44e5-b39b-f1c1182e7580 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:56.891312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd33a0-f579-499d-a344-a9d6e57f82c3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:56.932669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92780c9-53e7-46e5-a1bd-fd3f38861155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:56.933585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92780c9-53e7-46e5-a1bd-fd3f38861155 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:56.933864 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92780c9-53e7-46e5-a1bd-fd3f38861155 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:56.934137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92780c9-53e7-46e5-a1bd-fd3f38861155 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:56.934778 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92780c9-53e7-46e5-a1bd-fd3f38861155 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:56.936219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92780c9-53e7-46e5-a1bd-fd3f38861155 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:57.003927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff513f0-ef1f-44e5-b39b-f1c1182e7580 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:57.047827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 248/498] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:43:56 2022] POST /identity/v3/users => generated 254 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:43:57.050285 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c14e37-833e-4be9-a15e-1a9a981be579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:57.050938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c14e37-833e-4be9-a15e-1a9a981be579 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:57.051215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c14e37-833e-4be9-a15e-1a9a981be579 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:57.051486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c14e37-833e-4be9-a15e-1a9a981be579 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:57.060328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c14e37-833e-4be9-a15e-1a9a981be579 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:57.061952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c14e37-833e-4be9-a15e-1a9a981be579 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-50c14e37-833e-4be9-a15e-1a9a981be579 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:57.079799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:57.080804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:57.082350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 251/499] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:43:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:57.086729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92780c9-53e7-46e5-a1bd-fd3f38861155 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:57.141808 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:57.233655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92780c9-53e7-46e5-a1bd-fd3f38861155 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:57.274142 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92a3f3-7ee5-4042-acee-1bbafe6bfc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:57.274767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92a3f3-7ee5-4042-acee-1bbafe6bfc16 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:57.275048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92a3f3-7ee5-4042-acee-1bbafe6bfc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:57.275323 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92a3f3-7ee5-4042-acee-1bbafe6bfc16 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:57.276052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92a3f3-7ee5-4042-acee-1bbafe6bfc16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:57.276356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92a3f3-7ee5-4042-acee-1bbafe6bfc16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:57.278547 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92a3f3-7ee5-4042-acee-1bbafe6bfc16 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:57.300488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 249/500] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 12:43:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:57.305730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c14e37-833e-4be9-a15e-1a9a981be579 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:57.309325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:57.419996 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c14e37-833e-4be9-a15e-1a9a981be579 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:57.422459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2266c6-8cdd-446c-9470-a4cd241d61fa None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:57.422744 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2266c6-8cdd-446c-9470-a4cd241d61fa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:57.423020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2266c6-8cdd-446c-9470-a4cd241d61fa None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:57.424603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 250/501] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:57.430368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91c9ea-0fae-4dcd-b808-507e79731101 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:57.430989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91c9ea-0fae-4dcd-b808-507e79731101 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:57.431495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91c9ea-0fae-4dcd-b808-507e79731101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:57.456002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5d91c9ea-0fae-4dcd-b808-507e79731101 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:57.457504 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5d91c9ea-0fae-4dcd-b808-507e79731101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:57.462392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c15c5-fcd6-4898-b18a-0a742938590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:57.463003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c15c5-fcd6-4898-b18a-0a742938590a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:57.463280 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c15c5-fcd6-4898-b18a-0a742938590a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:57.463549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c15c5-fcd6-4898-b18a-0a742938590a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:57.575426 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 251/502] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:57.590320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c15c5-fcd6-4898-b18a-0a742938590a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:57.591786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c15c5-fcd6-4898-b18a-0a742938590a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:57.593654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:57.706587 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 252/503] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:43:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 15 12:43:57.714365 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91c9ea-0fae-4dcd-b808-507e79731101 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:57.746274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:57.845065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91c9ea-0fae-4dcd-b808-507e79731101 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:57.882834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d197f-df4b-42c6-9176-7982fbf266ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:57.883479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d197f-df4b-42c6-9176-7982fbf266ed None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:57.883752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d197f-df4b-42c6-9176-7982fbf266ed None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:57.884138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d197f-df4b-42c6-9176-7982fbf266ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:57.894942 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d197f-df4b-42c6-9176-7982fbf266ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:57.896396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d197f-df4b-42c6-9176-7982fbf266ed None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a27d197f-df4b-42c6-9176-7982fbf266ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:57.911981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:57.913452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:57.914333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:57.914333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:57.914333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:57.914333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 252/504] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:57.922460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c15c5-fcd6-4898-b18a-0a742938590a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:58.015333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d197f-df4b-42c6-9176-7982fbf266ed None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:58.055122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c15c5-fcd6-4898-b18a-0a742938590a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:58.057002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:58.090951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b59e62-77fc-44f6-8c49-2134624fbad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:58.091913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b59e62-77fc-44f6-8c49-2134624fbad6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.092417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b59e62-77fc-44f6-8c49-2134624fbad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.092889 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b59e62-77fc-44f6-8c49-2134624fbad6 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.093775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b59e62-77fc-44f6-8c49-2134624fbad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:58.094304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b59e62-77fc-44f6-8c49-2134624fbad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:58.095934 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b59e62-77fc-44f6-8c49-2134624fbad6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:58.108556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 253/505] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:58.114122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b59e62-77fc-44f6-8c49-2134624fbad6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:58.173525 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:58.214783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d197f-df4b-42c6-9176-7982fbf266ed None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:58.219865 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b59e62-77fc-44f6-8c49-2134624fbad6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:58.255251 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b733706-933b-4983-b1ec-46e915ed39fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:58.256244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b733706-933b-4983-b1ec-46e915ed39fc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.256725 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b733706-933b-4983-b1ec-46e915ed39fc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.257334 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b733706-933b-4983-b1ec-46e915ed39fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.270439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b733706-933b-4983-b1ec-46e915ed39fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:58.272359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b733706-933b-4983-b1ec-46e915ed39fc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:58.285883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41483fe-3f6a-4df6-8ad8-a3ba14a67bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:58.286839 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41483fe-3f6a-4df6-8ad8-a3ba14a67bfd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.287344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41483fe-3f6a-4df6-8ad8-a3ba14a67bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.287815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41483fe-3f6a-4df6-8ad8-a3ba14a67bfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6b733706-933b-4983-b1ec-46e915ed39fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:43:58.294618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:58.295600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:58.296500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:43:58.296500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:43:58.296500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:43:58.296500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:43:58.296500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:43:58.296500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:43:58.296500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:43:58.296500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:58.306769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 254/506] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:58.308735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b733706-933b-4983-b1ec-46e915ed39fc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:58.320070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41483fe-3f6a-4df6-8ad8-a3ba14a67bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e41483fe-3f6a-4df6-8ad8-a3ba14a67bfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:43:58.327693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:58.328642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:43:58.329620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:58.332231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 253/507] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:58.338726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41483fe-3f6a-4df6-8ad8-a3ba14a67bfd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:58.384136 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:58.487877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41483fe-3f6a-4df6-8ad8-a3ba14a67bfd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:58.536376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082f36a-bcce-48ac-b6b3-c66561c88c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:58.537397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082f36a-bcce-48ac-b6b3-c66561c88c75 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.537882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082f36a-bcce-48ac-b6b3-c66561c88c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.538354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082f36a-bcce-48ac-b6b3-c66561c88c75 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.539217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082f36a-bcce-48ac-b6b3-c66561c88c75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:58.539743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082f36a-bcce-48ac-b6b3-c66561c88c75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:58.541375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082f36a-bcce-48ac-b6b3-c66561c88c75 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:58.546719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:58.557787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 254/508] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:43:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:58.564482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082f36a-bcce-48ac-b6b3-c66561c88c75 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:58.575679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b733706-933b-4983-b1ec-46e915ed39fc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:58.616103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acefea9-6df3-45d0-8de3-2c9260ea4d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:58.617092 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acefea9-6df3-45d0-8de3-2c9260ea4d34 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.617603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acefea9-6df3-45d0-8de3-2c9260ea4d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.618130 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acefea9-6df3-45d0-8de3-2c9260ea4d34 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.619018 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acefea9-6df3-45d0-8de3-2c9260ea4d34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:58.619544 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acefea9-6df3-45d0-8de3-2c9260ea4d34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:58.621969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acefea9-6df3-45d0-8de3-2c9260ea4d34 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:58.636583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 255/509] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:43:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:58.642815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acefea9-6df3-45d0-8de3-2c9260ea4d34 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:58.784275 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:58.799321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082f36a-bcce-48ac-b6b3-c66561c88c75 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:58.821977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acefea9-6df3-45d0-8de3-2c9260ea4d34 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:58.860270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89826c55-cfa4-48cb-9f5a-85253ffe3e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:58.860911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89826c55-cfa4-48cb-9f5a-85253ffe3e2b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.861209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89826c55-cfa4-48cb-9f5a-85253ffe3e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.861475 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89826c55-cfa4-48cb-9f5a-85253ffe3e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.873529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff674d6-02b0-4a0e-ae2f-1ac96a27a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:58.873837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89826c55-cfa4-48cb-9f5a-85253ffe3e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:58.874653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff674d6-02b0-4a0e-ae2f-1ac96a27a0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.875149 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff674d6-02b0-4a0e-ae2f-1ac96a27a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.875267 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89826c55-cfa4-48cb-9f5a-85253ffe3e2b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:58.875930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff674d6-02b0-4a0e-ae2f-1ac96a27a0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.877172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff674d6-02b0-4a0e-ae2f-1ac96a27a0c6 None admin] RBAC: Adding query filter params (role.id=5a5e7aced6d84267bd9e1fa0c1016420, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=b849f213606a4686bfb140fec34d512f) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:58.878179 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff674d6-02b0-4a0e-ae2f-1ac96a27a0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:58.880123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff674d6-02b0-4a0e-ae2f-1ac96a27a0c6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:58.887966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 256/510] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:43:58 2022] GET /identity/v3/role_assignments?user.id=b849f213606a4686bfb140fec34d512f&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=5a5e7aced6d84267bd9e1fa0c1016420 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:58.888946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac3dad-d5c1-457f-a016-61f03a76bbab None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.889259 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac3dad-d5c1-457f-a016-61f03a76bbab None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.891637 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac3dad-d5c1-457f-a016-61f03a76bbab None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-89826c55-cfa4-48cb-9f5a-85253ffe3e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:58.891786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:58.893047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:58.894035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:58.894035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:58.894035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:58.894035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 255/511] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:58.894035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fdb57-7363-4ee8-8146-0924842088d6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.894035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fdb57-7363-4ee8-8146-0924842088d6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.894035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fdb57-7363-4ee8-8146-0924842088d6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.895055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 256/513] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:58.895152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 257/513] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:43:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:43:58.895880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dfe85-2b73-4675-8043-5ec3f664d218 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:58.896742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dfe85-2b73-4675-8043-5ec3f664d218 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:58.896742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dfe85-2b73-4675-8043-5ec3f664d218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:58.898560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac3dad-d5c1-457f-a016-61f03a76bbab None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:58.927458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ad2dfe85-2b73-4675-8043-5ec3f664d218 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:58.929863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ad2dfe85-2b73-4675-8043-5ec3f664d218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:58.989081 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:59.017729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 257/514] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:43:59.019489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af2642-4334-4999-9bb9-d95b42398a18 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:59.019999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af2642-4334-4999-9bb9-d95b42398a18 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:59.020481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af2642-4334-4999-9bb9-d95b42398a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:59.045138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-63af2642-4334-4999-9bb9-d95b42398a18 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:43:59.047521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-63af2642-4334-4999-9bb9-d95b42398a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:43:59.102815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac3dad-d5c1-457f-a016-61f03a76bbab None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:59.137329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 258/515] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:43:59.151772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f67ed-1ab4-4df9-b4ba-f8e3d44f7610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:59.152364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f67ed-1ab4-4df9-b4ba-f8e3d44f7610 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:59.152644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f67ed-1ab4-4df9-b4ba-f8e3d44f7610 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:59.152969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f67ed-1ab4-4df9-b4ba-f8e3d44f7610 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:59.153801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f67ed-1ab4-4df9-b4ba-f8e3d44f7610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:59.154112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f67ed-1ab4-4df9-b4ba-f8e3d44f7610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:59.155943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f67ed-1ab4-4df9-b4ba-f8e3d44f7610 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:59.169355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 258/516] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:59.175185 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af2642-4334-4999-9bb9-d95b42398a18 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:59.283178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af2642-4334-4999-9bb9-d95b42398a18 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:59.295592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:59.320300 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb541e2-699a-41a9-95bc-f3adaf2c650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:59.320931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb541e2-699a-41a9-95bc-f3adaf2c650b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:59.321219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb541e2-699a-41a9-95bc-f3adaf2c650b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:59.321503 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb541e2-699a-41a9-95bc-f3adaf2c650b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:59.328858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb541e2-699a-41a9-95bc-f3adaf2c650b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:59.330421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb541e2-699a-41a9-95bc-f3adaf2c650b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:59.334581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f67ed-1ab4-4df9-b4ba-f8e3d44f7610 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9fb541e2-699a-41a9-95bc-f3adaf2c650b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:43:59.347498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:59.348501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:43:59.349834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 259/517] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 12:43:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:59.351277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:59.355707 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb541e2-699a-41a9-95bc-f3adaf2c650b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:59.461705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f67ed-1ab4-4df9-b4ba-f8e3d44f7610 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:59.498034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb541e2-699a-41a9-95bc-f3adaf2c650b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:59.506638 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e817b-2ce3-4e14-9a81-815380533ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:59.507733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e817b-2ce3-4e14-9a81-815380533ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:59.508019 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e817b-2ce3-4e14-9a81-815380533ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:59.508298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e817b-2ce3-4e14-9a81-815380533ff4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:59.508890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e817b-2ce3-4e14-9a81-815380533ff4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:59.511536 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e817b-2ce3-4e14-9a81-815380533ff4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f73e817b-2ce3-4e14-9a81-815380533ff4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:59.525643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:59.526761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:59.528086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 259/518] 10.222.0.30 () {56 vars in 1129 bytes} [Sat Oct 15 12:43:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:43:59.529212 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e817b-2ce3-4e14-9a81-815380533ff4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:59.540035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88604571-bbc0-4bbb-9d9c-1c2cc0eddcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:59.540744 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88604571-bbc0-4bbb-9d9c-1c2cc0eddcf0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:59.541101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88604571-bbc0-4bbb-9d9c-1c2cc0eddcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:59.541470 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88604571-bbc0-4bbb-9d9c-1c2cc0eddcf0 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:59.542321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88604571-bbc0-4bbb-9d9c-1c2cc0eddcf0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:59.542709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88604571-bbc0-4bbb-9d9c-1c2cc0eddcf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:59.544237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88604571-bbc0-4bbb-9d9c-1c2cc0eddcf0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:59.573345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 260/519] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 12:43:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:43:59.574832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88604571-bbc0-4bbb-9d9c-1c2cc0eddcf0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:59.614643 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:59.624722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:59.666574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e817b-2ce3-4e14-9a81-815380533ff4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:59.706023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88604571-bbc0-4bbb-9d9c-1c2cc0eddcf0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:59.713954 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0123ed7-1324-490d-a967-ec480fa52f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:59.714819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0123ed7-1324-490d-a967-ec480fa52f26 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:59.715246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0123ed7-1324-490d-a967-ec480fa52f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:59.715685 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0123ed7-1324-490d-a967-ec480fa52f26 None admin] PATH_INFO: `/v3/services/network` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:59.716418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0123ed7-1324-490d-a967-ec480fa52f26 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:59.718249 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0123ed7-1324-490d-a967-ec480fa52f26 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f0123ed7-1324-490d-a967-ec480fa52f26 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:59.728626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:59.730409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 12:43:59.731557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:59.734063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 260/520] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:43:59 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:59.735172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0123ed7-1324-490d-a967-ec480fa52f26 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:59.750954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61a816-cd09-4742-98ec-a6ea7d99cdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:59.751577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61a816-cd09-4742-98ec-a6ea7d99cdba None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:59.751858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61a816-cd09-4742-98ec-a6ea7d99cdba None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:59.752136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61a816-cd09-4742-98ec-a6ea7d99cdba None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:59.752922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61a816-cd09-4742-98ec-a6ea7d99cdba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:59.753257 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61a816-cd09-4742-98ec-a6ea7d99cdba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:59.755042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61a816-cd09-4742-98ec-a6ea7d99cdba None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:59.765228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 261/521] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 12:43:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:59.765228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61a816-cd09-4742-98ec-a6ea7d99cdba None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:59.867855 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0123ed7-1324-490d-a967-ec480fa52f26 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:59.875219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:59.885716 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61a816-cd09-4742-98ec-a6ea7d99cdba None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:43:59.911729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fe638-6eae-4e1a-8496-3a8f415d4470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:59.912535 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fe638-6eae-4e1a-8496-3a8f415d4470 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:59.912919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fe638-6eae-4e1a-8496-3a8f415d4470 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:59.913223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fe638-6eae-4e1a-8496-3a8f415d4470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:59.915657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:43:59.927827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fe638-6eae-4e1a-8496-3a8f415d4470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:59.928857 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa19ffb-2476-4a7b-a3b7-35becbc7098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:43:59.929283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fe638-6eae-4e1a-8496-3a8f415d4470 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:59.929974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa19ffb-2476-4a7b-a3b7-35becbc7098e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:43:59.930493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa19ffb-2476-4a7b-a3b7-35becbc7098e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:43:59.930972 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa19ffb-2476-4a7b-a3b7-35becbc7098e None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:43:59.931802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa19ffb-2476-4a7b-a3b7-35becbc7098e None admin] RBAC: Adding query filter params (name=network) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:43:59.932323 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa19ffb-2476-4a7b-a3b7-35becbc7098e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:43:59.934301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa19ffb-2476-4a7b-a3b7-35becbc7098e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:43:59.939758 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 262/522] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:43:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:43:59.940853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa19ffb-2476-4a7b-a3b7-35becbc7098e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6a5fe638-6eae-4e1a-8496-3a8f415d4470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:43:59.946191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:43:59.947523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:43:59.948966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:43:59.948966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:43:59.948966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:43:59.950573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 261/523] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:43:59.951241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fe638-6eae-4e1a-8496-3a8f415d4470 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:00.051277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa19ffb-2476-4a7b-a3b7-35becbc7098e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:00.074730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fe638-6eae-4e1a-8496-3a8f415d4470 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:00.088045 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050caa-d4f1-4ece-9f0c-815ac623566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:00.088668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050caa-d4f1-4ece-9f0c-815ac623566c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:00.089524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050caa-d4f1-4ece-9f0c-815ac623566c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:00.089807 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050caa-d4f1-4ece-9f0c-815ac623566c None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:00.090452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39050caa-d4f1-4ece-9f0c-815ac623566c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:00.090752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39050caa-d4f1-4ece-9f0c-815ac623566c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:00.092137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39050caa-d4f1-4ece-9f0c-815ac623566c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:00.105234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 263/524] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 12:43:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:00.105234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050caa-d4f1-4ece-9f0c-815ac623566c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:00.123176 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44465af9-6f99-417d-ac6d-5e8e30c2c4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:00.123894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44465af9-6f99-417d-ac6d-5e8e30c2c4fc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:00.124248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44465af9-6f99-417d-ac6d-5e8e30c2c4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:00.124594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44465af9-6f99-417d-ac6d-5e8e30c2c4fc None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:00.125469 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44465af9-6f99-417d-ac6d-5e8e30c2c4fc None admin] RBAC: Adding query filter params (type=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:00.125984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44465af9-6f99-417d-ac6d-5e8e30c2c4fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:00.127372 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44465af9-6f99-417d-ac6d-5e8e30c2c4fc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:00.129531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:00.141417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 262/525] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:43:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:00.143125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e86a65-fa48-4e15-846d-73395ebe7249 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:00.143588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e86a65-fa48-4e15-846d-73395ebe7249 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:00.144021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e86a65-fa48-4e15-846d-73395ebe7249 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:00.145845 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 263/526] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:00.151494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aec8c-5a26-4e77-be2b-df32666422ac None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:00.152035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aec8c-5a26-4e77-be2b-df32666422ac None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:00.152462 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aec8c-5a26-4e77-be2b-df32666422ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:00.192826 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7d6aec8c-5a26-4e77-be2b-df32666422ac None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:00.194120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7d6aec8c-5a26-4e77-be2b-df32666422ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:00.300216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050caa-d4f1-4ece-9f0c-815ac623566c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:00.306834 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:00.347138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 264/527] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:44:00.362333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c23432-e6b6-4b6d-8e7a-7b8148c063da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:00.362970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c23432-e6b6-4b6d-8e7a-7b8148c063da None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:00.363222 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c23432-e6b6-4b6d-8e7a-7b8148c063da None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:00.363495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c23432-e6b6-4b6d-8e7a-7b8148c063da None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:00.364135 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c23432-e6b6-4b6d-8e7a-7b8148c063da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:00.364439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c23432-e6b6-4b6d-8e7a-7b8148c063da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:00.365895 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c23432-e6b6-4b6d-8e7a-7b8148c063da None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:00.378498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 264/528] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:00.383942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aec8c-5a26-4e77-be2b-df32666422ac None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:00.508302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aec8c-5a26-4e77-be2b-df32666422ac None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:00.550378 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54794e-f53e-4f8d-8bc1-c1d0d458df76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:00.551019 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54794e-f53e-4f8d-8bc1-c1d0d458df76 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:00.551300 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54794e-f53e-4f8d-8bc1-c1d0d458df76 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:00.551580 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54794e-f53e-4f8d-8bc1-c1d0d458df76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:00.575000 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54794e-f53e-4f8d-8bc1-c1d0d458df76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:00.576450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54794e-f53e-4f8d-8bc1-c1d0d458df76 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-fd54794e-f53e-4f8d-8bc1-c1d0d458df76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:00.591322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:00.592338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:00.593314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:00.593314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:00.593314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:00.593314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:00.593314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:00.593314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:00.593314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:00.593314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:00.593314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 265/529] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:00.598748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c23432-e6b6-4b6d-8e7a-7b8148c063da None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:00.697999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:00.719986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c23432-e6b6-4b6d-8e7a-7b8148c063da None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:00.734062 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54794e-f53e-4f8d-8bc1-c1d0d458df76 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:00.780330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc65fd-6862-438d-a7fa-ee01f95a12f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:00.781350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc65fd-6862-438d-a7fa-ee01f95a12f0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:00.781894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc65fd-6862-438d-a7fa-ee01f95a12f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:00.782397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc65fd-6862-438d-a7fa-ee01f95a12f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:00.783301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc65fd-6862-438d-a7fa-ee01f95a12f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:00.783813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc65fd-6862-438d-a7fa-ee01f95a12f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:00.786029 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc65fd-6862-438d-a7fa-ee01f95a12f0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:00.802240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 265/530] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:00.808573 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc65fd-6862-438d-a7fa-ee01f95a12f0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:00.814779 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:00.890464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54794e-f53e-4f8d-8bc1-c1d0d458df76 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:00.926086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03020772-4f79-43e7-9130-872d8514d532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:00.926724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03020772-4f79-43e7-9130-872d8514d532 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:00.927006 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03020772-4f79-43e7-9130-872d8514d532 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:00.927280 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03020772-4f79-43e7-9130-872d8514d532 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:00.938815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03020772-4f79-43e7-9130-872d8514d532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-03020772-4f79-43e7-9130-872d8514d532 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:00.944241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:00.945252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:00.946111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 266/531] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:44:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:00.951339 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03020772-4f79-43e7-9130-872d8514d532 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:01.000566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc65fd-6862-438d-a7fa-ee01f95a12f0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:01.073674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d94b1-f542-4bd4-b056-f8edfc6dea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:01.074420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d94b1-f542-4bd4-b056-f8edfc6dea7d None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:01.074701 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d94b1-f542-4bd4-b056-f8edfc6dea7d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:01.074979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d94b1-f542-4bd4-b056-f8edfc6dea7d None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/44b2b449acaa45f099c8a56ecbf20ca6/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:01.078019 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d94b1-f542-4bd4-b056-f8edfc6dea7d None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=44b2b449acaa45f099c8a56ecbf20ca6, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:01.079466 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d94b1-f542-4bd4-b056-f8edfc6dea7d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:01.116295 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:01.119931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03020772-4f79-43e7-9130-872d8514d532 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:01.157471 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9bb30-91bc-4c12-91e5-dcdac90998f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:01.157471 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9bb30-91bc-4c12-91e5-dcdac90998f5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:01.157471 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9bb30-91bc-4c12-91e5-dcdac90998f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:01.157471 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9bb30-91bc-4c12-91e5-dcdac90998f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:01.157871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9bb30-91bc-4c12-91e5-dcdac90998f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:01.157871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9bb30-91bc-4c12-91e5-dcdac90998f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:01.159196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9bb30-91bc-4c12-91e5-dcdac90998f5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:01.168719 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 267/532] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:44:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:01.174154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9bb30-91bc-4c12-91e5-dcdac90998f5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:01.196563 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:01.201517 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 266/533] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:44:00 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/44b2b449acaa45f099c8a56ecbf20ca6/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:44:01.321975 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9bb30-91bc-4c12-91e5-dcdac90998f5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:01.366601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067193a-a393-4151-9689-75f6ff427d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:01.367253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067193a-a393-4151-9689-75f6ff427d89 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:01.367533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067193a-a393-4151-9689-75f6ff427d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:01.367805 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067193a-a393-4151-9689-75f6ff427d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:01.382528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067193a-a393-4151-9689-75f6ff427d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:01.384048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067193a-a393-4151-9689-75f6ff427d89 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6067193a-a393-4151-9689-75f6ff427d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:01.402493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:01.403522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:01.404430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:01.404430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:01.404430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:01.406812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 268/534] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:01.414458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d94b1-f542-4bd4-b056-f8edfc6dea7d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:01.488838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cd61e-5dbd-4ca1-8412-f347e32dccb5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:01.489162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cd61e-5dbd-4ca1-8412-f347e32dccb5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:01.489433 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cd61e-5dbd-4ca1-8412-f347e32dccb5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:01.490752 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 269/535] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:01.496989 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557176a-6da1-4b2d-8f5e-82b81a92cb23 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:01.497311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557176a-6da1-4b2d-8f5e-82b81a92cb23 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:01.497645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557176a-6da1-4b2d-8f5e-82b81a92cb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:01.519494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4557176a-6da1-4b2d-8f5e-82b81a92cb23 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:01.520712 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4557176a-6da1-4b2d-8f5e-82b81a92cb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:01.543121 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:01.565008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d94b1-f542-4bd4-b056-f8edfc6dea7d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:01.615505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 270/536] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 12:44:01.624177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11fab3-08c6-4e03-9230-496056a10abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:01.624999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11fab3-08c6-4e03-9230-496056a10abd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:01.625456 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11fab3-08c6-4e03-9230-496056a10abd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:01.625875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11fab3-08c6-4e03-9230-496056a10abd None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:01.626671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11fab3-08c6-4e03-9230-496056a10abd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:01.627141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11fab3-08c6-4e03-9230-496056a10abd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:01.628695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11fab3-08c6-4e03-9230-496056a10abd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:01.648195 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 267/537] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:01.654497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557176a-6da1-4b2d-8f5e-82b81a92cb23 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:01.760174 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557176a-6da1-4b2d-8f5e-82b81a92cb23 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:01.796814 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697c731-a377-4006-8d19-f96cef48a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:01.797486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697c731-a377-4006-8d19-f96cef48a78d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:01.797765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697c731-a377-4006-8d19-f96cef48a78d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:01.798038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697c731-a377-4006-8d19-f96cef48a78d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:01.805386 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697c731-a377-4006-8d19-f96cef48a78d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:01.806962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697c731-a377-4006-8d19-f96cef48a78d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d697c731-a377-4006-8d19-f96cef48a78d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:01.819362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:01.820308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:01.821269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 271/538] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 12:44:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:01.828295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11fab3-08c6-4e03-9230-496056a10abd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:01.930501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:01.987819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11fab3-08c6-4e03-9230-496056a10abd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:02.021774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697c731-a377-4006-8d19-f96cef48a78d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:02.025599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:02.035785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988a905-5577-4de8-a3ee-066028e3897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:02.036406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988a905-5577-4de8-a3ee-066028e3897d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:02.041017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988a905-5577-4de8-a3ee-066028e3897d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:02.041017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988a905-5577-4de8-a3ee-066028e3897d None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:02.041017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988a905-5577-4de8-a3ee-066028e3897d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:02.041017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988a905-5577-4de8-a3ee-066028e3897d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:02.041017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988a905-5577-4de8-a3ee-066028e3897d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:02.064474 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 268/539] 10.222.0.30 () {56 vars in 1149 bytes} [Sat Oct 15 12:44:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:02.069740 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988a905-5577-4de8-a3ee-066028e3897d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:02.177943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697c731-a377-4006-8d19-f96cef48a78d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:02.202737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988a905-5577-4de8-a3ee-066028e3897d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:02.233560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:02.249836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ffd15-3483-43c6-9f92-add864ac5f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:02.250539 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ffd15-3483-43c6-9f92-add864ac5f7a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:02.250827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ffd15-3483-43c6-9f92-add864ac5f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:02.251101 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ffd15-3483-43c6-9f92-add864ac5f7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:02.265616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441eca4-edd7-4596-afbf-828164ca07ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:02.277417 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441eca4-edd7-4596-afbf-828164ca07ee None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:02.277417 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441eca4-edd7-4596-afbf-828164ca07ee None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:02.277417 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441eca4-edd7-4596-afbf-828164ca07ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:02.277685 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ffd15-3483-43c6-9f92-add864ac5f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c63ffd15-3483-43c6-9f92-add864ac5f7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:02.282702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:02.284063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:02.285227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 272/540] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:44:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:02.288722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441eca4-edd7-4596-afbf-828164ca07ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:02.290365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ffd15-3483-43c6-9f92-add864ac5f7a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:02.290851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441eca4-edd7-4596-afbf-828164ca07ee None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f441eca4-edd7-4596-afbf-828164ca07ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:02.308711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:02.312194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:02.313108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:02.313108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:02.313108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:02.313108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 269/541] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:02.316443 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441eca4-edd7-4596-afbf-828164ca07ee None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:02.429152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441eca4-edd7-4596-afbf-828164ca07ee None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:02.477488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:02.495875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a52612-7205-4833-8ddb-dcefd13a7221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:02.496499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a52612-7205-4833-8ddb-dcefd13a7221 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:02.496780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a52612-7205-4833-8ddb-dcefd13a7221 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:02.497053 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a52612-7205-4833-8ddb-dcefd13a7221 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:02.497726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a52612-7205-4833-8ddb-dcefd13a7221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:02.498033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a52612-7205-4833-8ddb-dcefd13a7221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:02.499408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a52612-7205-4833-8ddb-dcefd13a7221 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:02.527303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 270/542] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:02.548569 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a52612-7205-4833-8ddb-dcefd13a7221 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:02.562620 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ffd15-3483-43c6-9f92-add864ac5f7a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:02.601561 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:02.642279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e14aad-8ef9-4f05-8e1f-e3baffc95d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:02.642914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e14aad-8ef9-4f05-8e1f-e3baffc95d65 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:02.643275 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e14aad-8ef9-4f05-8e1f-e3baffc95d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:02.643556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e14aad-8ef9-4f05-8e1f-e3baffc95d65 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:02.644225 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e14aad-8ef9-4f05-8e1f-e3baffc95d65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:02.644601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e14aad-8ef9-4f05-8e1f-e3baffc95d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:02.646005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e14aad-8ef9-4f05-8e1f-e3baffc95d65 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:02.667678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 273/543] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:44:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:02.671623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e14aad-8ef9-4f05-8e1f-e3baffc95d65 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:02.724544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a52612-7205-4833-8ddb-dcefd13a7221 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:02.765065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4250d5-1871-4db0-99e4-df120548fa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:02.765747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4250d5-1871-4db0-99e4-df120548fa58 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:02.766028 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4250d5-1871-4db0-99e4-df120548fa58 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:02.766308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4250d5-1871-4db0-99e4-df120548fa58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:02.776143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4250d5-1871-4db0-99e4-df120548fa58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:02.778441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4250d5-1871-4db0-99e4-df120548fa58 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4f4250d5-1871-4db0-99e4-df120548fa58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:02.805700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:02.806615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:02.807526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:02.807526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:02.807526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:02.807526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:02.807526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:02.807526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:02.807526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:02.807526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:02.812704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 271/544] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:02.819253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc413f-0e4d-4fe7-ac0a-f2c167b58661 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:02.821152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc413f-0e4d-4fe7-ac0a-f2c167b58661 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:02.824039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc413f-0e4d-4fe7-ac0a-f2c167b58661 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:02.828002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 272/545] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:02.829545 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc413f-0e4d-4fe7-ac0a-f2c167b58661 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:02.864952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e14aad-8ef9-4f05-8e1f-e3baffc95d65 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:02.895733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:02.906013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0665c62-2f0f-4163-8670-30be26b983b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:02.906660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0665c62-2f0f-4163-8670-30be26b983b3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:02.906944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0665c62-2f0f-4163-8670-30be26b983b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:02.907223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0665c62-2f0f-4163-8670-30be26b983b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:02.919420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0665c62-2f0f-4163-8670-30be26b983b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:02.921007 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0665c62-2f0f-4163-8670-30be26b983b3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c0665c62-2f0f-4163-8670-30be26b983b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:02.937341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:02.938335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:02.939629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:02.939629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:02.939629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:02.939629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 274/546] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:02.940431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1fa0e-da81-46d0-93a4-4676c98f46e0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:02.940802 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1fa0e-da81-46d0-93a4-4676c98f46e0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:02.941173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1fa0e-da81-46d0-93a4-4676c98f46e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:02.955044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:02.962961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-66e1fa0e-da81-46d0-93a4-4676c98f46e0 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:02.970260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-66e1fa0e-da81-46d0-93a4-4676c98f46e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:02.970810 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc413f-0e4d-4fe7-ac0a-f2c167b58661 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:03.010524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95e77-1ec5-4735-b83a-08d8f578af89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:03.011221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95e77-1ec5-4735-b83a-08d8f578af89 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.011643 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95e77-1ec5-4735-b83a-08d8f578af89 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.011919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95e77-1ec5-4735-b83a-08d8f578af89 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.012614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95e77-1ec5-4735-b83a-08d8f578af89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:03.013342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95e77-1ec5-4735-b83a-08d8f578af89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:03.014790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95e77-1ec5-4735-b83a-08d8f578af89 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:03.027152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 273/547] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:03.028003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95e77-1ec5-4735-b83a-08d8f578af89 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:03.049491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 275/548] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:44:03.051093 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85040bc-5c0d-4d69-8fb7-a8161400bacf None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.051557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85040bc-5c0d-4d69-8fb7-a8161400bacf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.052022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85040bc-5c0d-4d69-8fb7-a8161400bacf None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.053453 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 276/549] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:03.054318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85040bc-5c0d-4d69-8fb7-a8161400bacf None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:03.166984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95e77-1ec5-4735-b83a-08d8f578af89 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:03.181358 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85040bc-5c0d-4d69-8fb7-a8161400bacf None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:03.192299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:03.203407 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89131a8b-ef01-4999-ad96-b19a71b691af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:03.204280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131a8b-ef01-4999-ad96-b19a71b691af None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.204775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131a8b-ef01-4999-ad96-b19a71b691af None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.205328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131a8b-ef01-4999-ad96-b19a71b691af None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.206177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89131a8b-ef01-4999-ad96-b19a71b691af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:03.206676 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89131a8b-ef01-4999-ad96-b19a71b691af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:03.208230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89131a8b-ef01-4999-ad96-b19a71b691af None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:03.219408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3efc2-860a-4b00-9e56-2b9de8547a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:03.220126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3efc2-860a-4b00-9e56-2b9de8547a84 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.220403 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3efc2-860a-4b00-9e56-2b9de8547a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.220675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3efc2-860a-4b00-9e56-2b9de8547a84 None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/16c94590fafa48f58fd94526a913dd66/roles/5a5e7aced6d84267bd9e1fa0c1016420` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.222396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 274/550] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:03.223895 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927d3da-e0d5-4617-9bb3-8c74c61e4c1c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.224448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927d3da-e0d5-4617-9bb3-8c74c61e4c1c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.224926 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927d3da-e0d5-4617-9bb3-8c74c61e4c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.237957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3efc2-860a-4b00-9e56-2b9de8547a84 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=16c94590fafa48f58fd94526a913dd66, role_id=5a5e7aced6d84267bd9e1fa0c1016420)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:03.239461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3efc2-860a-4b00-9e56-2b9de8547a84 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:03.246518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a927d3da-e0d5-4617-9bb3-8c74c61e4c1c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:03.247913 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a927d3da-e0d5-4617-9bb3-8c74c61e4c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:03.251020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:03.309366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 277/551] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:44:03 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/16c94590fafa48f58fd94526a913dd66/roles/5a5e7aced6d84267bd9e1fa0c1016420 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:44:03.310869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3efc2-860a-4b00-9e56-2b9de8547a84 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:03.324172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 275/552] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 12:44:03.341979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927d3da-e0d5-4617-9bb3-8c74c61e4c1c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:03.474751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3efc2-860a-4b00-9e56-2b9de8547a84 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:03.522543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17b57b-4d7c-45bc-addb-0b6eb22f13be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:03.523195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17b57b-4d7c-45bc-addb-0b6eb22f13be None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.523537 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17b57b-4d7c-45bc-addb-0b6eb22f13be None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.523870 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17b57b-4d7c-45bc-addb-0b6eb22f13be None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.532424 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17b57b-4d7c-45bc-addb-0b6eb22f13be None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:03.534062 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17b57b-4d7c-45bc-addb-0b6eb22f13be None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3e17b57b-4d7c-45bc-addb-0b6eb22f13be None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:03.550776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:03.551798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 12:44:03.552749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:03.554277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927d3da-e0d5-4617-9bb3-8c74c61e4c1c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:03.557406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 278/553] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:44:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:03.559009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17b57b-4d7c-45bc-addb-0b6eb22f13be None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:03.581524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:03.642033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b21d4a-c986-4d69-aea9-3697af15b405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:03.643060 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b21d4a-c986-4d69-aea9-3697af15b405 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.643544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b21d4a-c986-4d69-aea9-3697af15b405 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.644013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b21d4a-c986-4d69-aea9-3697af15b405 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.644890 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b21d4a-c986-4d69-aea9-3697af15b405 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:03.653212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b21d4a-c986-4d69-aea9-3697af15b405 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:03.676852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17b57b-4d7c-45bc-addb-0b6eb22f13be None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:03.711982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec717327-8a4b-4d7e-bea0-d179f03a438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:03.712667 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec717327-8a4b-4d7e-bea0-d179f03a438f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.712946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec717327-8a4b-4d7e-bea0-d179f03a438f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.713269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec717327-8a4b-4d7e-bea0-d179f03a438f None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.714148 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec717327-8a4b-4d7e-bea0-d179f03a438f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:03.714699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec717327-8a4b-4d7e-bea0-d179f03a438f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:03.716141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec717327-8a4b-4d7e-bea0-d179f03a438f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:03.735167 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 279/554] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 12:44:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:03.735904 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec717327-8a4b-4d7e-bea0-d179f03a438f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:03.740652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:03.746439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 276/555] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:44:03 2022] POST /identity/v3/services => generated 240 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:44:03.748146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6ebad-b909-4396-bcfc-3870de7e34d3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.748533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6ebad-b909-4396-bcfc-3870de7e34d3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.748870 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6ebad-b909-4396-bcfc-3870de7e34d3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.754380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 277/556] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:03.754380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6ebad-b909-4396-bcfc-3870de7e34d3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:03.866077 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:03.877895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec717327-8a4b-4d7e-bea0-d179f03a438f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:03.881931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6ebad-b909-4396-bcfc-3870de7e34d3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:03.923600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aea654-ff56-457f-818d-e94ccab96182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:03.924372 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aea654-ff56-457f-818d-e94ccab96182 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.924656 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aea654-ff56-457f-818d-e94ccab96182 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.924932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aea654-ff56-457f-818d-e94ccab96182 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.925649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aea654-ff56-457f-818d-e94ccab96182 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:03.927300 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aea654-ff56-457f-818d-e94ccab96182 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:03.933176 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d39d8c-3a3c-4020-94e3-8a1a483eb0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:03.934889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d39d8c-3a3c-4020-94e3-8a1a483eb0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.935240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d39d8c-3a3c-4020-94e3-8a1a483eb0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.935591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d39d8c-3a3c-4020-94e3-8a1a483eb0b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.947987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d39d8c-3a3c-4020-94e3-8a1a483eb0b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:03.950039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d39d8c-3a3c-4020-94e3-8a1a483eb0b5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d8d39d8c-3a3c-4020-94e3-8a1a483eb0b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:03.966212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:03.967264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:03.968246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:03.968246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:03.968246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:03.968246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 278/557] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:03.968530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1010060-97d7-46d7-8d42-b1c4aa2099af None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:03.969036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1010060-97d7-46d7-8d42-b1c4aa2099af None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:03.969036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1010060-97d7-46d7-8d42-b1c4aa2099af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:03.989254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f1010060-97d7-46d7-8d42-b1c4aa2099af None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:03.990368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f1010060-97d7-46d7-8d42-b1c4aa2099af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:04.008314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 280/558] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:44:03 2022] POST /identity/v3/services => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:04.017341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aea654-ff56-457f-818d-e94ccab96182 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:04.066474 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:04.132185 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 279/559] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:04.220444 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aea654-ff56-457f-818d-e94ccab96182 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:04.256821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64365eea-a4db-4f56-8687-2c881217091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:04.257387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64365eea-a4db-4f56-8687-2c881217091d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:04.257668 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64365eea-a4db-4f56-8687-2c881217091d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:04.257942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64365eea-a4db-4f56-8687-2c881217091d None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:04.258643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64365eea-a4db-4f56-8687-2c881217091d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:04.259014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64365eea-a4db-4f56-8687-2c881217091d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:04.260420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64365eea-a4db-4f56-8687-2c881217091d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:04.263365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:04.274046 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 281/560] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:04.280144 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1010060-97d7-46d7-8d42-b1c4aa2099af None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:04.401965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1010060-97d7-46d7-8d42-b1c4aa2099af None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:04.438621 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f912f-4fe1-4d4d-90ec-12f4a67b01f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:04.439254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f912f-4fe1-4d4d-90ec-12f4a67b01f9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:04.439531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f912f-4fe1-4d4d-90ec-12f4a67b01f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:04.439802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f912f-4fe1-4d4d-90ec-12f4a67b01f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:04.446536 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f912f-4fe1-4d4d-90ec-12f4a67b01f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:04.448000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f912f-4fe1-4d4d-90ec-12f4a67b01f9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-863f912f-4fe1-4d4d-90ec-12f4a67b01f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:04.459859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:04.460835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:04.461787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:04.461787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:04.461787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:04.461787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:04.461787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:04.461787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:04.461787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:04.461787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:04.461787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 280/561] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:04.468498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64365eea-a4db-4f56-8687-2c881217091d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:04.474264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f912f-4fe1-4d4d-90ec-12f4a67b01f9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:04.511771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:04.622870 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f912f-4fe1-4d4d-90ec-12f4a67b01f9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:04.662837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64365eea-a4db-4f56-8687-2c881217091d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:04.694637 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35254ee1-0771-4c27-b748-c81967161693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:04.695329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35254ee1-0771-4c27-b748-c81967161693 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:04.695610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35254ee1-0771-4c27-b748-c81967161693 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:04.695881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35254ee1-0771-4c27-b748-c81967161693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:04.714489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35254ee1-0771-4c27-b748-c81967161693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-35254ee1-0771-4c27-b748-c81967161693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:04.723853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:04.724871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:04.725798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:04.732021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 281/562] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:04.733714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952deda-d9dd-4e90-98b9-9984aedcd13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:04.734416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952deda-d9dd-4e90-98b9-9984aedcd13a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:04.734749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952deda-d9dd-4e90-98b9-9984aedcd13a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:04.735026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952deda-d9dd-4e90-98b9-9984aedcd13a None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:04.735720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952deda-d9dd-4e90-98b9-9984aedcd13a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:04.736027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952deda-d9dd-4e90-98b9-9984aedcd13a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:04.737499 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952deda-d9dd-4e90-98b9-9984aedcd13a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:04.740019 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35254ee1-0771-4c27-b748-c81967161693 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:04.758531 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 282/563] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:04.766263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952deda-d9dd-4e90-98b9-9984aedcd13a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:04.878031 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:04.920097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952deda-d9dd-4e90-98b9-9984aedcd13a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:04.923312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:04.938303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35254ee1-0771-4c27-b748-c81967161693 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:04.964029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4091f6-dfb3-4b18-ae8f-359823bb9dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:04.968341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4091f6-dfb3-4b18-ae8f-359823bb9dc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:04.969016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4091f6-dfb3-4b18-ae8f-359823bb9dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:04.969330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4091f6-dfb3-4b18-ae8f-359823bb9dc2 None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/b849f213606a4686bfb140fec34d512f/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:04.974472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4091f6-dfb3-4b18-ae8f-359823bb9dc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=b849f213606a4686bfb140fec34d512f, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:04.975915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4091f6-dfb3-4b18-ae8f-359823bb9dc2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:04.994157 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff307b9-8087-4845-9d9b-d28549442e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:04.997321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff307b9-8087-4845-9d9b-d28549442e9b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:04.997321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff307b9-8087-4845-9d9b-d28549442e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:04.997321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff307b9-8087-4845-9d9b-d28549442e9b None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:04.997321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff307b9-8087-4845-9d9b-d28549442e9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:04.997321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff307b9-8087-4845-9d9b-d28549442e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:04.998048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff307b9-8087-4845-9d9b-d28549442e9b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:05.007280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 282/564] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:44:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:05.013332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff307b9-8087-4845-9d9b-d28549442e9b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:05.095451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 283/565] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:44:04 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/b849f213606a4686bfb140fec34d512f/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:44:05.128450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff307b9-8087-4845-9d9b-d28549442e9b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:05.168576 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed03854-cfa8-49a9-93f5-755e08ad3a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:05.169688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed03854-cfa8-49a9-93f5-755e08ad3a59 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:05.170176 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed03854-cfa8-49a9-93f5-755e08ad3a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:05.170659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed03854-cfa8-49a9-93f5-755e08ad3a59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:05.182495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed03854-cfa8-49a9-93f5-755e08ad3a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:05.184683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed03854-cfa8-49a9-93f5-755e08ad3a59 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8ed03854-cfa8-49a9-93f5-755e08ad3a59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:05.201269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:05.202288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:05.203181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:05.203181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:05.203181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:05.205514 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 283/566] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:05.211856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4091f6-dfb3-4b18-ae8f-359823bb9dc2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:05.357909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4091f6-dfb3-4b18-ae8f-359823bb9dc2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:05.381128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:05.399026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10da32-7a2f-425b-9ae6-e4ffc2ad261d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:05.399646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10da32-7a2f-425b-9ae6-e4ffc2ad261d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:05.399928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10da32-7a2f-425b-9ae6-e4ffc2ad261d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:05.400204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10da32-7a2f-425b-9ae6-e4ffc2ad261d None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:05.400847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10da32-7a2f-425b-9ae6-e4ffc2ad261d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:05.401155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10da32-7a2f-425b-9ae6-e4ffc2ad261d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:05.403085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10da32-7a2f-425b-9ae6-e4ffc2ad261d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:05.415903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 284/567] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:05.421729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed03854-cfa8-49a9-93f5-755e08ad3a59 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:05.437728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:05.538202 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed03854-cfa8-49a9-93f5-755e08ad3a59 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:05.587307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d9bef-8e79-4a73-91e9-f1ffd3e520ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:05.587950 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d9bef-8e79-4a73-91e9-f1ffd3e520ab None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:05.588235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d9bef-8e79-4a73-91e9-f1ffd3e520ab None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:05.588514 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d9bef-8e79-4a73-91e9-f1ffd3e520ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:05.600760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d9bef-8e79-4a73-91e9-f1ffd3e520ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:05.602242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d9bef-8e79-4a73-91e9-f1ffd3e520ab None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-1d5d9bef-8e79-4a73-91e9-f1ffd3e520ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:05.616189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:05.617178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:05.618181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 284/568] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 12:44:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:05.623821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10da32-7a2f-425b-9ae6-e4ffc2ad261d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:05.814858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10da32-7a2f-425b-9ae6-e4ffc2ad261d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:05.857056 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa27e12-faee-4afe-bc44-093938e6b287 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:05.857709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa27e12-faee-4afe-bc44-093938e6b287 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:05.857709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa27e12-faee-4afe-bc44-093938e6b287 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:05.859385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 285/569] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:05.866040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409c39a-8056-4f11-88a2-8faf4cf4a82a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:05.866615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409c39a-8056-4f11-88a2-8faf4cf4a82a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:05.867152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409c39a-8056-4f11-88a2-8faf4cf4a82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:05.867341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92f4d7-a50f-44c2-b1cc-df8b8c80a273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:05.867951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92f4d7-a50f-44c2-b1cc-df8b8c80a273 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:05.868234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92f4d7-a50f-44c2-b1cc-df8b8c80a273 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:05.868508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92f4d7-a50f-44c2-b1cc-df8b8c80a273 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:05.869265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92f4d7-a50f-44c2-b1cc-df8b8c80a273 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:05.869579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92f4d7-a50f-44c2-b1cc-df8b8c80a273 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:05.871069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92f4d7-a50f-44c2-b1cc-df8b8c80a273 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:05.903481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a409c39a-8056-4f11-88a2-8faf4cf4a82a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:05.906643 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a409c39a-8056-4f11-88a2-8faf4cf4a82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:05.908499 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 285/570] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 12:44:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:05.917605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92f4d7-a50f-44c2-b1cc-df8b8c80a273 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:05.968178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:06.023494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92f4d7-a50f-44c2-b1cc-df8b8c80a273 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:06.060320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910d196-7e33-4701-bdb2-72cba97d51ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:06.060918 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d196-7e33-4701-bdb2-72cba97d51ab None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:06.061244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d196-7e33-4701-bdb2-72cba97d51ab None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:06.061517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d196-7e33-4701-bdb2-72cba97d51ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:06.073574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:06.079846 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 286/571] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:06.133567 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910d196-7e33-4701-bdb2-72cba97d51ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:06.135116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910d196-7e33-4701-bdb2-72cba97d51ab None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:06.169301 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:06.272036 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 286/572] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:44:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 12:44:06.277943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409c39a-8056-4f11-88a2-8faf4cf4a82a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:06.305678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49625577-7e15-4155-9405-4b967338c83f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:06.306013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49625577-7e15-4155-9405-4b967338c83f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:06.306240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49625577-7e15-4155-9405-4b967338c83f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:06.308628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 287/573] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:06.336058 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eae30-5583-404f-a528-c2e5818c58aa None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:06.336388 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eae30-5583-404f-a528-c2e5818c58aa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:06.336916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eae30-5583-404f-a528-c2e5818c58aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:06.372895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bc6eae30-5583-404f-a528-c2e5818c58aa None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:06.374120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bc6eae30-5583-404f-a528-c2e5818c58aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:06.412918 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409c39a-8056-4f11-88a2-8faf4cf4a82a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:06.461922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978a3c9-876a-4ebe-937a-b4281a6bf318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:06.462564 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a3c9-876a-4ebe-937a-b4281a6bf318 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:06.463551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a3c9-876a-4ebe-937a-b4281a6bf318 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:06.463877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a3c9-876a-4ebe-937a-b4281a6bf318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:06.483823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978a3c9-876a-4ebe-937a-b4281a6bf318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:06.485306 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978a3c9-876a-4ebe-937a-b4281a6bf318 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:06.506942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 288/574] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7978a3c9-876a-4ebe-937a-b4281a6bf318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:06.516187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:06.517213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:06.518128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:06.518128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:06.518128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:06.518128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 287/575] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:06.522879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eae30-5583-404f-a528-c2e5818c58aa None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:06.537160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978a3c9-876a-4ebe-937a-b4281a6bf318 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:06.618311 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:06.620561 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:06.693842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eae30-5583-404f-a528-c2e5818c58aa None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:06.696362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978a3c9-876a-4ebe-937a-b4281a6bf318 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:06.731135 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a52b3-b948-40d3-87ba-66122b51a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:06.731776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a52b3-b948-40d3-87ba-66122b51a488 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:06.732058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a52b3-b948-40d3-87ba-66122b51a488 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:06.732332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a52b3-b948-40d3-87ba-66122b51a488 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:06.742876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a52b3-b948-40d3-87ba-66122b51a488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9e4a52b3-b948-40d3-87ba-66122b51a488 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:06.748827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:06.749812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:06.750681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 288/576] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:44:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:06.755593 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a52b3-b948-40d3-87ba-66122b51a488 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:06.780911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa127845-19bf-4a33-855b-a75ef5a73b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:06.781679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa127845-19bf-4a33-855b-a75ef5a73b90 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:06.781960 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa127845-19bf-4a33-855b-a75ef5a73b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:06.782232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa127845-19bf-4a33-855b-a75ef5a73b90 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:06.782897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa127845-19bf-4a33-855b-a75ef5a73b90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:06.783204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa127845-19bf-4a33-855b-a75ef5a73b90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:06.785531 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa127845-19bf-4a33-855b-a75ef5a73b90 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:06.813507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 289/577] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:06.814732 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa127845-19bf-4a33-855b-a75ef5a73b90 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:06.908012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a52b3-b948-40d3-87ba-66122b51a488 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:06.922748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:06.946688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064e12d-026e-4068-9777-3ef9ad5ff799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:06.947258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064e12d-026e-4068-9777-3ef9ad5ff799 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:06.947543 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064e12d-026e-4068-9777-3ef9ad5ff799 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:06.947818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064e12d-026e-4068-9777-3ef9ad5ff799 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:06.948380 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa127845-19bf-4a33-855b-a75ef5a73b90 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:06.948820 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064e12d-026e-4068-9777-3ef9ad5ff799 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:06.948899 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064e12d-026e-4068-9777-3ef9ad5ff799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:06.950296 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064e12d-026e-4068-9777-3ef9ad5ff799 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:06.964291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 289/578] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:44:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:06.964291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064e12d-026e-4068-9777-3ef9ad5ff799 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:06.977923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:06.989819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef2bd4-725d-43ec-9dc5-857bc9a40bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:06.990445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef2bd4-725d-43ec-9dc5-857bc9a40bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:06.990895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef2bd4-725d-43ec-9dc5-857bc9a40bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:06.991172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef2bd4-725d-43ec-9dc5-857bc9a40bf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:06.998152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef2bd4-725d-43ec-9dc5-857bc9a40bf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:06.999667 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef2bd4-725d-43ec-9dc5-857bc9a40bf6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0aef2bd4-725d-43ec-9dc5-857bc9a40bf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:07.011812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:07.012857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:07.014049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:07.014049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:07.014049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:07.014049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:07.014049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:07.014049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:07.014049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:07.014049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:07.017558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 290/579] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:07.018636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef00a6c-4400-44fa-8135-e6b36df65bbf None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.018920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef00a6c-4400-44fa-8135-e6b36df65bbf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.019220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef00a6c-4400-44fa-8135-e6b36df65bbf None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.020548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 291/580] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:07.025377 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef00a6c-4400-44fa-8135-e6b36df65bbf None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.069018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064e12d-026e-4068-9777-3ef9ad5ff799 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.103844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b09c9-6352-45b4-9132-a7b7db75c4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.105240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b09c9-6352-45b4-9132-a7b7db75c4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.105554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b09c9-6352-45b4-9132-a7b7db75c4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.105862 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b09c9-6352-45b4-9132-a7b7db75c4b5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.106443 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b09c9-6352-45b4-9132-a7b7db75c4b5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.107826 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b09c9-6352-45b4-9132-a7b7db75c4b5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ab7b09c9-6352-45b4-9132-a7b7db75c4b5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:07.118847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:07.119996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:07.121687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 290/581] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:06 2022] GET /identity/v3/services/network => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:07.122507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d2296-ffe2-4e35-8e86-5584b3de9304 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.122507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d2296-ffe2-4e35-8e86-5584b3de9304 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.122507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d2296-ffe2-4e35-8e86-5584b3de9304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.139278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef00a6c-4400-44fa-8135-e6b36df65bbf None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.140693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7f8d2296-ffe2-4e35-8e86-5584b3de9304 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:07.141979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7f8d2296-ffe2-4e35-8e86-5584b3de9304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:07.171420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:07.174792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21cb2d-7d21-4630-bca5-ecd02b93e9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.181919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21cb2d-7d21-4630-bca5-ecd02b93e9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.181919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21cb2d-7d21-4630-bca5-ecd02b93e9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.181919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21cb2d-7d21-4630-bca5-ecd02b93e9b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.188463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21cb2d-7d21-4630-bca5-ecd02b93e9b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.189973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21cb2d-7d21-4630-bca5-ecd02b93e9b4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cb21cb2d-7d21-4630-bca5-ecd02b93e9b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:07.204731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:07.205709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:07.206586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:07.206586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:07.206586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:07.206586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 292/582] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:07.206586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21cb2d-7d21-4630-bca5-ecd02b93e9b4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.215299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 291/583] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:07.216206 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d2296-ffe2-4e35-8e86-5584b3de9304 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.251618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:07.329350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21cb2d-7d21-4630-bca5-ecd02b93e9b4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.338272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d2296-ffe2-4e35-8e86-5584b3de9304 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.375373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ffaa7-9d2c-4efc-820b-13bb54a2dffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.376265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ffaa7-9d2c-4efc-820b-13bb54a2dffd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.376557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ffaa7-9d2c-4efc-820b-13bb54a2dffd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.376839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ffaa7-9d2c-4efc-820b-13bb54a2dffd None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.377748 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ffaa7-9d2c-4efc-820b-13bb54a2dffd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:07.378067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ffaa7-9d2c-4efc-820b-13bb54a2dffd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.379799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ffaa7-9d2c-4efc-820b-13bb54a2dffd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.381459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b34d3-e0ec-4b72-8611-209a4e880ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.382454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b34d3-e0ec-4b72-8611-209a4e880ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.383244 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b34d3-e0ec-4b72-8611-209a4e880ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.383719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b34d3-e0ec-4b72-8611-209a4e880ae5 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.384548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b34d3-e0ec-4b72-8611-209a4e880ae5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:07.385024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b34d3-e0ec-4b72-8611-209a4e880ae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.387397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b34d3-e0ec-4b72-8611-209a4e880ae5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.393588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 292/584] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:07.394666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b34d3-e0ec-4b72-8611-209a4e880ae5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.403474 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 293/585] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:07.404648 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ffaa7-9d2c-4efc-820b-13bb54a2dffd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.432366 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:07.504846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:07.509891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b34d3-e0ec-4b72-8611-209a4e880ae5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.510296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ffaa7-9d2c-4efc-820b-13bb54a2dffd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.546562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4d55a-11bd-4e77-826d-2c3f578bed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.547175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4d55a-11bd-4e77-826d-2c3f578bed56 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.547452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4d55a-11bd-4e77-826d-2c3f578bed56 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.547720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4d55a-11bd-4e77-826d-2c3f578bed56 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.548348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4d55a-11bd-4e77-826d-2c3f578bed56 None admin] RBAC: Adding query filter params (type=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:07.548654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4d55a-11bd-4e77-826d-2c3f578bed56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.550248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4d55a-11bd-4e77-826d-2c3f578bed56 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.555353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5baa04b-6260-46f6-bae5-d0492697b37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.555353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baa04b-6260-46f6-bae5-d0492697b37c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.555353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baa04b-6260-46f6-bae5-d0492697b37c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.555637 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 294/586] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:07.556145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4d55a-11bd-4e77-826d-2c3f578bed56 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.557356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baa04b-6260-46f6-bae5-d0492697b37c None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.558136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5baa04b-6260-46f6-bae5-d0492697b37c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:07.558912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5baa04b-6260-46f6-bae5-d0492697b37c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.559963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5baa04b-6260-46f6-bae5-d0492697b37c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.576339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 293/587] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:07.576339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5baa04b-6260-46f6-bae5-d0492697b37c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.665483 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4d55a-11bd-4e77-826d-2c3f578bed56 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.674688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:07.686482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5baa04b-6260-46f6-bae5-d0492697b37c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.707898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d10da3-4b94-4403-a536-d2b6ebedeca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.708511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d10da3-4b94-4403-a536-d2b6ebedeca7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.709226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d10da3-4b94-4403-a536-d2b6ebedeca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.709226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d10da3-4b94-4403-a536-d2b6ebedeca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.709929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d10da3-4b94-4403-a536-d2b6ebedeca7 None admin] RBAC: Adding query filter params (role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=44b2b449acaa45f099c8a56ecbf20ca6) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:07.710235 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d10da3-4b94-4403-a536-d2b6ebedeca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.711900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d10da3-4b94-4403-a536-d2b6ebedeca7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.719560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 295/588] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/role_assignments?user.id=44b2b449acaa45f099c8a56ecbf20ca6&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:07.719560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d10da3-4b94-4403-a536-d2b6ebedeca7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.734903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a53315-728b-46e5-b7bb-1e3925363c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.735682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a53315-728b-46e5-b7bb-1e3925363c45 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.736041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a53315-728b-46e5-b7bb-1e3925363c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.736394 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a53315-728b-46e5-b7bb-1e3925363c45 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.737071 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a53315-728b-46e5-b7bb-1e3925363c45 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.738732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a53315-728b-46e5-b7bb-1e3925363c45 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d7a53315-728b-46e5-b7bb-1e3925363c45 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:07.749467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:07.750515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:07.751356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 294/589] 10.222.0.30 () {56 vars in 1129 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:07.752933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a53315-728b-46e5-b7bb-1e3925363c45 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.764869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:07.855710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d10da3-4b94-4403-a536-d2b6ebedeca7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.873989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a53315-728b-46e5-b7bb-1e3925363c45 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:07.894635 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11c415-22d8-4149-9d33-fa33a03b5491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.895752 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11c415-22d8-4149-9d33-fa33a03b5491 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.896049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11c415-22d8-4149-9d33-fa33a03b5491 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.896325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11c415-22d8-4149-9d33-fa33a03b5491 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.897047 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11c415-22d8-4149-9d33-fa33a03b5491 None admin] RBAC: Adding query filter params (interface=public, service_id=f17e9f82c8474635b38c05ff1126ed53, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:07.897391 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11c415-22d8-4149-9d33-fa33a03b5491 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.898791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11c415-22d8-4149-9d33-fa33a03b5491 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.910453 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 296/590] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/endpoints?service_id=f17e9f82c8474635b38c05ff1126ed53&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:07.911215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11c415-22d8-4149-9d33-fa33a03b5491 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.915840 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b994d21-fd63-492d-bcba-b3ebb878fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:07.916506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b994d21-fd63-492d-bcba-b3ebb878fb69 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.916822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b994d21-fd63-492d-bcba-b3ebb878fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.917965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b994d21-fd63-492d-bcba-b3ebb878fb69 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.927280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b994d21-fd63-492d-bcba-b3ebb878fb69 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:07.928739 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b994d21-fd63-492d-bcba-b3ebb878fb69 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9b994d21-fd63-492d-bcba-b3ebb878fb69 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:07.941600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:07.942617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:07.943583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 295/591] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:07.945871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0eea9b-491d-44f4-a8c2-82b14a6ad3d3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:07.945871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0eea9b-491d-44f4-a8c2-82b14a6ad3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:07.946274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0eea9b-491d-44f4-a8c2-82b14a6ad3d3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:07.947696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 296/592] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:07.948600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0eea9b-491d-44f4-a8c2-82b14a6ad3d3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:07.965343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:08.023091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:08.037151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11c415-22d8-4149-9d33-fa33a03b5491 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:08.059293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0eea9b-491d-44f4-a8c2-82b14a6ad3d3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:08.073722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da7819-11eb-4fcd-a5c3-9f44f50fc88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:08.074332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da7819-11eb-4fcd-a5c3-9f44f50fc88d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:08.074610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da7819-11eb-4fcd-a5c3-9f44f50fc88d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:08.074882 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da7819-11eb-4fcd-a5c3-9f44f50fc88d None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:08.075514 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da7819-11eb-4fcd-a5c3-9f44f50fc88d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:08.075825 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da7819-11eb-4fcd-a5c3-9f44f50fc88d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:08.077250 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da7819-11eb-4fcd-a5c3-9f44f50fc88d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:08.081664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 297/593] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:08.082449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da7819-11eb-4fcd-a5c3-9f44f50fc88d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:08.104847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd6283-a6c0-42f2-b0a9-63c5e35e4821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:08.105501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd6283-a6c0-42f2-b0a9-63c5e35e4821 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:08.105777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd6283-a6c0-42f2-b0a9-63c5e35e4821 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:08.106049 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd6283-a6c0-42f2-b0a9-63c5e35e4821 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:08.106648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd6283-a6c0-42f2-b0a9-63c5e35e4821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:08.108031 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd6283-a6c0-42f2-b0a9-63c5e35e4821 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:08.113286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 297/594] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 12:44:07 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:08.114437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd4a96-92c1-419e-8718-4d71b8df5f99 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:08.114723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd4a96-92c1-419e-8718-4d71b8df5f99 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:08.114996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd4a96-92c1-419e-8718-4d71b8df5f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:08.134386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e5cd4a96-92c1-419e-8718-4d71b8df5f99 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:08.136376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e5cd4a96-92c1-419e-8718-4d71b8df5f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:08.210901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da7819-11eb-4fcd-a5c3-9f44f50fc88d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:08.230969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 298/595] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:08.233079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd4a96-92c1-419e-8718-4d71b8df5f99 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:08.261082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:08.280426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6e152-02d9-4fb8-a352-22a89a31517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:08.282244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6e152-02d9-4fb8-a352-22a89a31517e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:08.282804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6e152-02d9-4fb8-a352-22a89a31517e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:08.286979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6e152-02d9-4fb8-a352-22a89a31517e None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:08.288014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6e152-02d9-4fb8-a352-22a89a31517e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:08.288497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6e152-02d9-4fb8-a352-22a89a31517e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:08.293713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6e152-02d9-4fb8-a352-22a89a31517e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:08.325627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 298/596] 10.222.0.30 () {56 vars in 1149 bytes} [Sat Oct 15 12:44:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:08.331026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6e152-02d9-4fb8-a352-22a89a31517e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:08.353109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:08.413874 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd4a96-92c1-419e-8718-4d71b8df5f99 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:08.455991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6e152-02d9-4fb8-a352-22a89a31517e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:08.461684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59dea3-55ef-4db0-84b6-3d2c1fe556dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:08.462342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59dea3-55ef-4db0-84b6-3d2c1fe556dc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:08.462624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59dea3-55ef-4db0-84b6-3d2c1fe556dc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:08.462897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59dea3-55ef-4db0-84b6-3d2c1fe556dc None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:08.463530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59dea3-55ef-4db0-84b6-3d2c1fe556dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:08.464530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59dea3-55ef-4db0-84b6-3d2c1fe556dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:08.466502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59dea3-55ef-4db0-84b6-3d2c1fe556dc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:08.473925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 299/597] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 12:44:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:08.479770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59dea3-55ef-4db0-84b6-3d2c1fe556dc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:08.538002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356ee15-5bba-4ad5-a8af-0c89a5eebe66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:08.538568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356ee15-5bba-4ad5-a8af-0c89a5eebe66 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:08.538852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356ee15-5bba-4ad5-a8af-0c89a5eebe66 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:08.539131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356ee15-5bba-4ad5-a8af-0c89a5eebe66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:08.569489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:08.603754 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59dea3-55ef-4db0-84b6-3d2c1fe556dc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:08.622212 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356ee15-5bba-4ad5-a8af-0c89a5eebe66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:08.623857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356ee15-5bba-4ad5-a8af-0c89a5eebe66 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:08.636665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:08.649222 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f628cd-e95a-41c0-94f7-00393e424f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:08.650189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f628cd-e95a-41c0-94f7-00393e424f7f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:08.650474 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f628cd-e95a-41c0-94f7-00393e424f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:08.650876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f628cd-e95a-41c0-94f7-00393e424f7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:08.652532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f628cd-e95a-41c0-94f7-00393e424f7f None admin] RBAC: Adding query filter params (interface=public, service_id=cddbf0c58dbf4ae3a0b7630f625fb789, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:08.653009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f628cd-e95a-41c0-94f7-00393e424f7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:08.655078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f628cd-e95a-41c0-94f7-00393e424f7f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:08.669291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 300/598] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 12:44:08 2022] GET /identity/v3/endpoints?service_id=cddbf0c58dbf4ae3a0b7630f625fb789&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:08.670222 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f628cd-e95a-41c0-94f7-00393e424f7f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:08.744578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 299/599] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:44:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 12:44:08.746558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356ee15-5bba-4ad5-a8af-0c89a5eebe66 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:08.807156 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f628cd-e95a-41c0-94f7-00393e424f7f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:08.858149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c83061-089c-4f01-9c63-a86cc91640df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:08.859016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356ee15-5bba-4ad5-a8af-0c89a5eebe66 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:08.859411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c83061-089c-4f01-9c63-a86cc91640df None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:08.859892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c83061-089c-4f01-9c63-a86cc91640df None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:08.860625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c83061-089c-4f01-9c63-a86cc91640df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:08.864606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:08.872616 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:08.877678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c83061-089c-4f01-9c63-a86cc91640df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-83c83061-089c-4f01-9c63-a86cc91640df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:08.882558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:08.883771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:08.884663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 301/600] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:44:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:08.885420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c83061-089c-4f01-9c63-a86cc91640df None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:08.904804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c365f-23bc-4b81-9575-e7b063075978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:08.905616 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c365f-23bc-4b81-9575-e7b063075978 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:08.905910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c365f-23bc-4b81-9575-e7b063075978 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:08.906187 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c365f-23bc-4b81-9575-e7b063075978 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:08.906794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c365f-23bc-4b81-9575-e7b063075978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:08.908709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c365f-23bc-4b81-9575-e7b063075978 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:08.919923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 300/601] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 12:44:08 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:08.919923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c365f-23bc-4b81-9575-e7b063075978 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:09.041235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c83061-089c-4f01-9c63-a86cc91640df None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:09.070775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eaedf-afa5-4e8e-8c54-6cf8e5845d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:09.071350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eaedf-afa5-4e8e-8c54-6cf8e5845d6c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:09.071630 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eaedf-afa5-4e8e-8c54-6cf8e5845d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:09.071905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eaedf-afa5-4e8e-8c54-6cf8e5845d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:09.084866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eaedf-afa5-4e8e-8c54-6cf8e5845d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:09.086297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eaedf-afa5-4e8e-8c54-6cf8e5845d6c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:09.105746 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c365f-23bc-4b81-9575-e7b063075978 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4b5eaedf-afa5-4e8e-8c54-6cf8e5845d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:09.123631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:09.124691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:09.125650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:09.125650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:09.125650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:09.133715 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 302/602] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:09.142280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eaedf-afa5-4e8e-8c54-6cf8e5845d6c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:09.155563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b9948-6ce9-4038-bb41-b32d790d0153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:09.156492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b9948-6ce9-4038-bb41-b32d790d0153 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:09.156965 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b9948-6ce9-4038-bb41-b32d790d0153 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:09.157478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b9948-6ce9-4038-bb41-b32d790d0153 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:09.158482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b9948-6ce9-4038-bb41-b32d790d0153 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:09.158961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b9948-6ce9-4038-bb41-b32d790d0153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:09.160780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b9948-6ce9-4038-bb41-b32d790d0153 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:09.171668 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 301/603] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:44:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:09.177542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b9948-6ce9-4038-bb41-b32d790d0153 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:09.218629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:09.226390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:09.311050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b9948-6ce9-4038-bb41-b32d790d0153 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:09.327428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eaedf-afa5-4e8e-8c54-6cf8e5845d6c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:09.347348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6c46c-f7f6-480f-8e55-bbf4ba937977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:09.347980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6c46c-f7f6-480f-8e55-bbf4ba937977 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:09.348260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6c46c-f7f6-480f-8e55-bbf4ba937977 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:09.348530 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6c46c-f7f6-480f-8e55-bbf4ba937977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:09.359481 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6c46c-f7f6-480f-8e55-bbf4ba937977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:09.362625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6c46c-f7f6-480f-8e55-bbf4ba937977 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7bf6c46c-f7f6-480f-8e55-bbf4ba937977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:09.376246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:09.377422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:09.378342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:09.378342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:09.378342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:09.378342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 302/604] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:09.385028 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6c46c-f7f6-480f-8e55-bbf4ba937977 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:09.397737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f2023-6ee3-4b65-9f94-8cd93b706903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:09.398353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f2023-6ee3-4b65-9f94-8cd93b706903 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:09.398630 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f2023-6ee3-4b65-9f94-8cd93b706903 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:09.398948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f2023-6ee3-4b65-9f94-8cd93b706903 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:09.399587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f2023-6ee3-4b65-9f94-8cd93b706903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:09.399887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f2023-6ee3-4b65-9f94-8cd93b706903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:09.403362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f2023-6ee3-4b65-9f94-8cd93b706903 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:09.430520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 303/605] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:09.440364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f2023-6ee3-4b65-9f94-8cd93b706903 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:09.511047 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:09.523417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6c46c-f7f6-480f-8e55-bbf4ba937977 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:09.572094 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d3919-5d51-40d3-b51c-a6eda9de02b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:09.572707 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d3919-5d51-40d3-b51c-a6eda9de02b4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:09.573588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d3919-5d51-40d3-b51c-a6eda9de02b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:09.574429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d3919-5d51-40d3-b51c-a6eda9de02b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:09.574539 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d3919-5d51-40d3-b51c-a6eda9de02b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:09.574829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d3919-5d51-40d3-b51c-a6eda9de02b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:09.576191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d3919-5d51-40d3-b51c-a6eda9de02b4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:09.589609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 303/606] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:09.595305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d3919-5d51-40d3-b51c-a6eda9de02b4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:09.650469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f2023-6ee3-4b65-9f94-8cd93b706903 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:09.671147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:09.702916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d3919-5d51-40d3-b51c-a6eda9de02b4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:09.738548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55cefe-3d17-458d-bbee-7643a2c8467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:09.739233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55cefe-3d17-458d-bbee-7643a2c8467d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:09.739559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55cefe-3d17-458d-bbee-7643a2c8467d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:09.739835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55cefe-3d17-458d-bbee-7643a2c8467d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:09.741901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a89124-3e5a-4458-b69d-21dfb23b692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:09.742551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a89124-3e5a-4458-b69d-21dfb23b692e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:09.742838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a89124-3e5a-4458-b69d-21dfb23b692e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:09.743117 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a89124-3e5a-4458-b69d-21dfb23b692e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:09.747512 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55cefe-3d17-458d-bbee-7643a2c8467d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:09.748953 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55cefe-3d17-458d-bbee-7643a2c8467d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:09.754459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a89124-3e5a-4458-b69d-21dfb23b692e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:09.755919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a89124-3e5a-4458-b69d-21dfb23b692e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1c55cefe-3d17-458d-bbee-7643a2c8467d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:09.761799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:09.762736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:09.763708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 304/607] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:44:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-08a89124-3e5a-4458-b69d-21dfb23b692e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:09.772748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:09.774002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:09.775260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:09.775260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:09.775260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:09.775260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:09.775260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:09.775260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:09.775260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:09.775260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:09.775782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55cefe-3d17-458d-bbee-7643a2c8467d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:09.775879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 304/608] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:09.782094 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a89124-3e5a-4458-b69d-21dfb23b692e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:09.796322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:09.963070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55cefe-3d17-458d-bbee-7643a2c8467d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:09.966090 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a89124-3e5a-4458-b69d-21dfb23b692e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:10.017498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40888033-dfce-45b5-a2d6-e02117c3c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:10.018115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40888033-dfce-45b5-a2d6-e02117c3c40f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.018398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40888033-dfce-45b5-a2d6-e02117c3c40f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.018669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40888033-dfce-45b5-a2d6-e02117c3c40f None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.019369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40888033-dfce-45b5-a2d6-e02117c3c40f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:10.019676 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40888033-dfce-45b5-a2d6-e02117c3c40f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:10.021121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40888033-dfce-45b5-a2d6-e02117c3c40f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:10.034427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 305/609] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:10.038027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467df47-7305-453e-88ea-e766696cfddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:10.038650 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467df47-7305-453e-88ea-e766696cfddc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.038931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467df47-7305-453e-88ea-e766696cfddc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.039209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467df47-7305-453e-88ea-e766696cfddc None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.039951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467df47-7305-453e-88ea-e766696cfddc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:10.040243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467df47-7305-453e-88ea-e766696cfddc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:10.042291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467df47-7305-453e-88ea-e766696cfddc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:10.043064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40888033-dfce-45b5-a2d6-e02117c3c40f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:10.084788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 305/610] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 12:44:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:10.089553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:10.092350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467df47-7305-453e-88ea-e766696cfddc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:10.193527 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:10.197169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40888033-dfce-45b5-a2d6-e02117c3c40f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:10.237965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b323c-77ef-4acf-9500-57f424983d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:10.238499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b323c-77ef-4acf-9500-57f424983d39 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.238777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b323c-77ef-4acf-9500-57f424983d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.239055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b323c-77ef-4acf-9500-57f424983d39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.239799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b323c-77ef-4acf-9500-57f424983d39 None admin] RBAC: Adding query filter params (role.id=5a5e7aced6d84267bd9e1fa0c1016420, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=16c94590fafa48f58fd94526a913dd66) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:10.240094 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b323c-77ef-4acf-9500-57f424983d39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:10.241552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b323c-77ef-4acf-9500-57f424983d39 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:10.248492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 306/611] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:44:10 2022] GET /identity/v3/role_assignments?user.id=16c94590fafa48f58fd94526a913dd66&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=5a5e7aced6d84267bd9e1fa0c1016420 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:10.274424 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467df47-7305-453e-88ea-e766696cfddc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:10.313070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d5bad-deee-4d7d-8bee-61c869dfafb9 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.313408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d5bad-deee-4d7d-8bee-61c869dfafb9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.313659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d5bad-deee-4d7d-8bee-61c869dfafb9 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.315162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 307/612] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:10.315916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1ff70-ba97-4399-b307-79e8f3da4cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:10.316625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1ff70-ba97-4399-b307-79e8f3da4cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.316817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1ff70-ba97-4399-b307-79e8f3da4cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.317074 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1ff70-ba97-4399-b307-79e8f3da4cf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.321844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7ef93-7bbc-48ae-a74e-bc54bea1c6d5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.322443 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7ef93-7bbc-48ae-a74e-bc54bea1c6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.323461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7ef93-7bbc-48ae-a74e-bc54bea1c6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.343444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6ea7ef93-7bbc-48ae-a74e-bc54bea1c6d5 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:10.345116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6ea7ef93-7bbc-48ae-a74e-bc54bea1c6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:10.399005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1ff70-ba97-4399-b307-79e8f3da4cf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:10.400513 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1ff70-ba97-4399-b307-79e8f3da4cf6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:10.478919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:10.514713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 308/613] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:10.519754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 306/614] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:44:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 15 12:44:10.526399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7ef93-7bbc-48ae-a74e-bc54bea1c6d5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:10.566470 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:10.651632 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7ef93-7bbc-48ae-a74e-bc54bea1c6d5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:10.690855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ac27b-f8aa-4027-9050-1f2695f3f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:10.691486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ac27b-f8aa-4027-9050-1f2695f3f586 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.691763 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ac27b-f8aa-4027-9050-1f2695f3f586 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.692029 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ac27b-f8aa-4027-9050-1f2695f3f586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.702563 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ac27b-f8aa-4027-9050-1f2695f3f586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:10.704017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ac27b-f8aa-4027-9050-1f2695f3f586 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f88ac27b-f8aa-4027-9050-1f2695f3f586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:10.719295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:10.720261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:10.721150 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:10.721150 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:10.721150 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:10.721150 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 309/615] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:10.727385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1ff70-ba97-4399-b307-79e8f3da4cf6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:10.788021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f1fd9-c953-4d0f-b20c-3d0ec3df710a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.788667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f1fd9-c953-4d0f-b20c-3d0ec3df710a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.789137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f1fd9-c953-4d0f-b20c-3d0ec3df710a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.791050 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 310/616] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:10.806568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208ddb1-dde4-4b35-a650-ff54a9ff0130 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.807161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208ddb1-dde4-4b35-a650-ff54a9ff0130 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.808214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208ddb1-dde4-4b35-a650-ff54a9ff0130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.843288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0208ddb1-dde4-4b35-a650-ff54a9ff0130 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:10.844600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0208ddb1-dde4-4b35-a650-ff54a9ff0130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:10.867727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1ff70-ba97-4399-b307-79e8f3da4cf6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:10.914120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a489df4-a2aa-47ce-a23f-ca5cb6fc34e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:10.914844 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a489df4-a2aa-47ce-a23f-ca5cb6fc34e2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:10.915124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a489df4-a2aa-47ce-a23f-ca5cb6fc34e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:10.915396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a489df4-a2aa-47ce-a23f-ca5cb6fc34e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:10.916037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a489df4-a2aa-47ce-a23f-ca5cb6fc34e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:10.916341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a489df4-a2aa-47ce-a23f-ca5cb6fc34e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:10.918452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a489df4-a2aa-47ce-a23f-ca5cb6fc34e2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:10.919095 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 311/617] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Oct 15 12:44:10.919941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208ddb1-dde4-4b35-a650-ff54a9ff0130 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:10.936369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:10.946310 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 307/618] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:10.953124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a489df4-a2aa-47ce-a23f-ca5cb6fc34e2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:10.992278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:11.119294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208ddb1-dde4-4b35-a650-ff54a9ff0130 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:11.120234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a489df4-a2aa-47ce-a23f-ca5cb6fc34e2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:11.156275 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892140e-07d7-4b2d-81dd-808b25eb5b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:11.156915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892140e-07d7-4b2d-81dd-808b25eb5b7a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.157219 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892140e-07d7-4b2d-81dd-808b25eb5b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.157511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892140e-07d7-4b2d-81dd-808b25eb5b7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.165380 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892140e-07d7-4b2d-81dd-808b25eb5b7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:11.167366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892140e-07d7-4b2d-81dd-808b25eb5b7a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0892140e-07d7-4b2d-81dd-808b25eb5b7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:11.179958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:11.180861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:11.182155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:11.182155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:11.182155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:11.182155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:11.182155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:11.182155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:11.182155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:11.182155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:11.182155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 308/619] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:11.183761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704255c-a383-4727-b2ba-0d368eda75ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:11.184623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704255c-a383-4727-b2ba-0d368eda75ab None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.185107 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704255c-a383-4727-b2ba-0d368eda75ab None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.185619 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704255c-a383-4727-b2ba-0d368eda75ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.186403 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704255c-a383-4727-b2ba-0d368eda75ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:11.187216 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892140e-07d7-4b2d-81dd-808b25eb5b7a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:11.188283 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704255c-a383-4727-b2ba-0d368eda75ab None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a704255c-a383-4727-b2ba-0d368eda75ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:11.210151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:11.211141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:11.212055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 312/620] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 12:44:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:11.234309 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704255c-a383-4727-b2ba-0d368eda75ab None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:11.292431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:11.314897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892140e-07d7-4b2d-81dd-808b25eb5b7a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:11.331013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:11.363200 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72043b33-7b09-4634-a75a-d6ea67e55bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:11.364474 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72043b33-7b09-4634-a75a-d6ea67e55bdc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.365210 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72043b33-7b09-4634-a75a-d6ea67e55bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.365723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72043b33-7b09-4634-a75a-d6ea67e55bdc None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.366619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72043b33-7b09-4634-a75a-d6ea67e55bdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:11.366794 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704255c-a383-4727-b2ba-0d368eda75ab None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:11.367284 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72043b33-7b09-4634-a75a-d6ea67e55bdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:11.369539 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72043b33-7b09-4634-a75a-d6ea67e55bdc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:11.387273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 309/621] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:11.388548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72043b33-7b09-4634-a75a-d6ea67e55bdc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:11.406089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db4b12-1c98-40a8-a123-7bdddce8b716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:11.406715 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db4b12-1c98-40a8-a123-7bdddce8b716 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.407002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db4b12-1c98-40a8-a123-7bdddce8b716 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.407279 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db4b12-1c98-40a8-a123-7bdddce8b716 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.407912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db4b12-1c98-40a8-a123-7bdddce8b716 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:11.408219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db4b12-1c98-40a8-a123-7bdddce8b716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:11.410011 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db4b12-1c98-40a8-a123-7bdddce8b716 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:11.419713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 313/622] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:11.420623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db4b12-1c98-40a8-a123-7bdddce8b716 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:11.522909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72043b33-7b09-4634-a75a-d6ea67e55bdc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:11.549800 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db4b12-1c98-40a8-a123-7bdddce8b716 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:11.563664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3370b-5fc8-4f6f-a72d-2914e0ddbd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:11.564614 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3370b-5fc8-4f6f-a72d-2914e0ddbd4c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.565141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3370b-5fc8-4f6f-a72d-2914e0ddbd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.565704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3370b-5fc8-4f6f-a72d-2914e0ddbd4c None admin] PATH_INFO: `/v3/services/network` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.566580 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3370b-5fc8-4f6f-a72d-2914e0ddbd4c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:11.568199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3370b-5fc8-4f6f-a72d-2914e0ddbd4c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-84f3370b-5fc8-4f6f-a72d-2914e0ddbd4c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:11.579106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:11.580160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 12:44:11.581030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:11.585724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 310/623] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:11 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:11.586381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:11.586585 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3370b-5fc8-4f6f-a72d-2914e0ddbd4c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:11.592777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e9942-194d-4abc-a384-7307875c783e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:11.593424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e9942-194d-4abc-a384-7307875c783e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.593710 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e9942-194d-4abc-a384-7307875c783e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.593987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e9942-194d-4abc-a384-7307875c783e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.594711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e9942-194d-4abc-a384-7307875c783e None admin] RBAC: Adding query filter params (role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=b849f213606a4686bfb140fec34d512f) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:11.595024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e9942-194d-4abc-a384-7307875c783e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:11.596875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e9942-194d-4abc-a384-7307875c783e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:11.599853 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:11.604281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 314/624] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:44:11 2022] GET /identity/v3/role_assignments?user.id=b849f213606a4686bfb140fec34d512f&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:11.605880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9aa73-8192-4f56-be41-c38d8558e54b None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.606199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9aa73-8192-4f56-be41-c38d8558e54b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.606444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9aa73-8192-4f56-be41-c38d8558e54b None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.609357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 315/625] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:11.609357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9aa73-8192-4f56-be41-c38d8558e54b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:11.747525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3370b-5fc8-4f6f-a72d-2914e0ddbd4c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:11.753697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9aa73-8192-4f56-be41-c38d8558e54b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:11.802156 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a70ce6-fe15-457c-9f1e-15936d3071fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:11.803092 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a70ce6-fe15-457c-9f1e-15936d3071fa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.803382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a70ce6-fe15-457c-9f1e-15936d3071fa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.803660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a70ce6-fe15-457c-9f1e-15936d3071fa None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.804298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a70ce6-fe15-457c-9f1e-15936d3071fa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:11.804608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a70ce6-fe15-457c-9f1e-15936d3071fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:11.805656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1675128-f749-482f-9af0-f73f1b13355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:11.806017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a70ce6-fe15-457c-9f1e-15936d3071fa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:11.806270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1675128-f749-482f-9af0-f73f1b13355a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.806546 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1675128-f749-482f-9af0-f73f1b13355a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.806815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1675128-f749-482f-9af0-f73f1b13355a None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.807949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1675128-f749-482f-9af0-f73f1b13355a None admin] RBAC: Adding query filter params (name=network) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:11.808428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1675128-f749-482f-9af0-f73f1b13355a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:11.810446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1675128-f749-482f-9af0-f73f1b13355a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:11.812011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 311/626] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:11.817179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 316/627] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:11.819145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84caa8-2c17-4d1f-9d30-2793ff19b5c8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:11.819442 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84caa8-2c17-4d1f-9d30-2793ff19b5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:11.819725 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84caa8-2c17-4d1f-9d30-2793ff19b5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:11.824497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1675128-f749-482f-9af0-f73f1b13355a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:11.848890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2c84caa8-2c17-4d1f-9d30-2793ff19b5c8 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:11.850190 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2c84caa8-2c17-4d1f-9d30-2793ff19b5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:11.931980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 312/628] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:11.985209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:12.104063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1675128-f749-482f-9af0-f73f1b13355a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:12.146347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9dc83-9fb0-4d4e-b6db-2d3cbb3f4634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:12.146969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9dc83-9fb0-4d4e-b6db-2d3cbb3f4634 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:12.153212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9dc83-9fb0-4d4e-b6db-2d3cbb3f4634 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:12.153212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9dc83-9fb0-4d4e-b6db-2d3cbb3f4634 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:12.153212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9dc83-9fb0-4d4e-b6db-2d3cbb3f4634 None admin] RBAC: Adding query filter params (type=network) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:12.153212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9dc83-9fb0-4d4e-b6db-2d3cbb3f4634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:12.153212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9dc83-9fb0-4d4e-b6db-2d3cbb3f4634 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:12.158982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 317/629] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:12.163969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84caa8-2c17-4d1f-9d30-2793ff19b5c8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:12.177883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:12.271725 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9dc83-9fb0-4d4e-b6db-2d3cbb3f4634 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:12.345140 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84caa8-2c17-4d1f-9d30-2793ff19b5c8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:12.458067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e0ce8-baf2-489c-a214-4f8c8304d686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:12.458637 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e0ce8-baf2-489c-a214-4f8c8304d686 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:12.458923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e0ce8-baf2-489c-a214-4f8c8304d686 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:12.459205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e0ce8-baf2-489c-a214-4f8c8304d686 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:12.459901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e0ce8-baf2-489c-a214-4f8c8304d686 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:12.461309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e0ce8-baf2-489c-a214-4f8c8304d686 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:12.471465 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9dc83-9fb0-4d4e-b6db-2d3cbb3f4634 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:12.519452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:12.545366 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48115127-b853-4bc7-a765-58da874d38fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:12.546025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48115127-b853-4bc7-a765-58da874d38fd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:12.553626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48115127-b853-4bc7-a765-58da874d38fd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:12.553626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48115127-b853-4bc7-a765-58da874d38fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:12.558413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48115127-b853-4bc7-a765-58da874d38fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:12.558413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48115127-b853-4bc7-a765-58da874d38fd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-48115127-b853-4bc7-a765-58da874d38fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:12.580116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:12.582427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 12:44:12.583329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:12.586271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 318/630] 10.222.0.30 () {56 vars in 1129 bytes} [Sat Oct 15 12:44:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:12.590708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48115127-b853-4bc7-a765-58da874d38fd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:12.597220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 313/631] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 12:44:12 2022] POST /identity/v3/endpoints => generated 337 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:12.770757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48115127-b853-4bc7-a765-58da874d38fd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:12.813073 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb57ee-3617-4b0c-9698-19bbf15b2901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:12.813724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb57ee-3617-4b0c-9698-19bbf15b2901 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:12.814003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb57ee-3617-4b0c-9698-19bbf15b2901 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:12.814273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb57ee-3617-4b0c-9698-19bbf15b2901 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:12.814962 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb57ee-3617-4b0c-9698-19bbf15b2901 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:12.815267 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb57ee-3617-4b0c-9698-19bbf15b2901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:12.816634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb57ee-3617-4b0c-9698-19bbf15b2901 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:12.821522 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 319/632] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 12:44:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:12.827486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e0ce8-baf2-489c-a214-4f8c8304d686 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:12.926570 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:12.981445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e0ce8-baf2-489c-a214-4f8c8304d686 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:13.027050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d6bcb-887b-4570-9cb6-e1a64931ff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:13.027684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6bcb-887b-4570-9cb6-e1a64931ff08 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:13.027961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6bcb-887b-4570-9cb6-e1a64931ff08 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:13.028230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6bcb-887b-4570-9cb6-e1a64931ff08 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:13.028881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d6bcb-887b-4570-9cb6-e1a64931ff08 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:13.029204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d6bcb-887b-4570-9cb6-e1a64931ff08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:13.030577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d6bcb-887b-4570-9cb6-e1a64931ff08 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:13.036435 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 314/633] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 12:44:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:13.042546 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb57ee-3617-4b0c-9698-19bbf15b2901 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:13.094972 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:13.106777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7308dd9-ed5b-4d7c-803b-54bfb117f068 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:13.107073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7308dd9-ed5b-4d7c-803b-54bfb117f068 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:13.107358 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7308dd9-ed5b-4d7c-803b-54bfb117f068 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:13.110138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 315/634] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:13.114146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c759b7-afdf-448a-939c-d12c5b1e6f97 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:13.114436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c759b7-afdf-448a-939c-d12c5b1e6f97 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:13.114718 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c759b7-afdf-448a-939c-d12c5b1e6f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:13.134346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-94c759b7-afdf-448a-939c-d12c5b1e6f97 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:13.135620 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-94c759b7-afdf-448a-939c-d12c5b1e6f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:13.156335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb57ee-3617-4b0c-9698-19bbf15b2901 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:13.200356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d255532-6080-444a-b8d1-5840a62b43ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:13.200976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d255532-6080-444a-b8d1-5840a62b43ed None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:13.201673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d255532-6080-444a-b8d1-5840a62b43ed None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:13.201967 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d255532-6080-444a-b8d1-5840a62b43ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:13.202637 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d255532-6080-444a-b8d1-5840a62b43ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:13.204043 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d255532-6080-444a-b8d1-5840a62b43ed None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:13.221411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 316/635] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:13.282591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 320/636] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 12:44:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:13.504919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c759b7-afdf-448a-939c-d12c5b1e6f97 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:13.571720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:13.620849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c759b7-afdf-448a-939c-d12c5b1e6f97 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:13.668157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c28580-baf9-4140-bfc7-f43f493b1108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:13.668797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28580-baf9-4140-bfc7-f43f493b1108 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:13.669074 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28580-baf9-4140-bfc7-f43f493b1108 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:13.669383 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28580-baf9-4140-bfc7-f43f493b1108 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:13.680587 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c28580-baf9-4140-bfc7-f43f493b1108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f5c28580-baf9-4140-bfc7-f43f493b1108 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:13.686163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:13.687073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:13.688279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 317/637] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 12:44:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:13.693345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d255532-6080-444a-b8d1-5840a62b43ed None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:13.799013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d255532-6080-444a-b8d1-5840a62b43ed None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:13.801386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:13.837026 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b0d27-f955-45b6-801b-5f35d58c659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:13.837684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b0d27-f955-45b6-801b-5f35d58c659d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:13.837969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b0d27-f955-45b6-801b-5f35d58c659d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:13.838234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b0d27-f955-45b6-801b-5f35d58c659d None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:13.839493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b0d27-f955-45b6-801b-5f35d58c659d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:13.839829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b0d27-f955-45b6-801b-5f35d58c659d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:13.841363 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b0d27-f955-45b6-801b-5f35d58c659d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:13.850058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 321/638] 10.222.0.30 () {56 vars in 1115 bytes} [Sat Oct 15 12:44:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:13.856304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c28580-baf9-4140-bfc7-f43f493b1108 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:13.968989 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c28580-baf9-4140-bfc7-f43f493b1108 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:13.980695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:14.002128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace172ad-574a-48fa-ab57-31ba895951f7 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.002684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace172ad-574a-48fa-ab57-31ba895951f7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.003047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace172ad-574a-48fa-ab57-31ba895951f7 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.004750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 322/639] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:14.009880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502179a-5686-43e8-ac53-735f7988b148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:14.011061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502179a-5686-43e8-ac53-735f7988b148 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.011385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502179a-5686-43e8-ac53-735f7988b148 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.011575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529516b-05d7-4ba6-af52-46e901dd9975 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.011672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502179a-5686-43e8-ac53-735f7988b148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.011990 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529516b-05d7-4ba6-af52-46e901dd9975 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.012270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529516b-05d7-4ba6-af52-46e901dd9975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.024409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502179a-5686-43e8-ac53-735f7988b148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:14.026533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502179a-5686-43e8-ac53-735f7988b148 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:14.030784 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b529516b-05d7-4ba6-af52-46e901dd9975 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:14.032064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b529516b-05d7-4ba6-af52-46e901dd9975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f502179a-5686-43e8-ac53-735f7988b148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:14.042332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:14.043293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:14.044463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:14.044463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:14.044463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:14.044463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 318/640] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:14.048944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502179a-5686-43e8-ac53-735f7988b148 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:14.109028 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 323/641] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:14.165071 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502179a-5686-43e8-ac53-735f7988b148 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:14.201118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47efa8-38b9-45f1-8139-d858df6a6df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:14.201761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47efa8-38b9-45f1-8139-d858df6a6df2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.202040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47efa8-38b9-45f1-8139-d858df6a6df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.202357 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47efa8-38b9-45f1-8139-d858df6a6df2 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.202998 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47efa8-38b9-45f1-8139-d858df6a6df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:14.203302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47efa8-38b9-45f1-8139-d858df6a6df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:14.204660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47efa8-38b9-45f1-8139-d858df6a6df2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:14.218280 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 319/642] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:14.226285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529516b-05d7-4ba6-af52-46e901dd9975 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:14.328412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:14.335429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529516b-05d7-4ba6-af52-46e901dd9975 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:14.376600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d225972-9420-4c60-a43a-562ece395e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:14.377153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d225972-9420-4c60-a43a-562ece395e58 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.377457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d225972-9420-4c60-a43a-562ece395e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.377728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d225972-9420-4c60-a43a-562ece395e58 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.385296 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d225972-9420-4c60-a43a-562ece395e58 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:14.387445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d225972-9420-4c60-a43a-562ece395e58 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:14.395188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47efa8-38b9-45f1-8139-d858df6a6df2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9d225972-9420-4c60-a43a-562ece395e58 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:14.400575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:14.402286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:14.403842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 324/643] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 12:44:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:14.407958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d225972-9420-4c60-a43a-562ece395e58 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:14.422959 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:14.505623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47efa8-38b9-45f1-8139-d858df6a6df2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:14.519870 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d225972-9420-4c60-a43a-562ece395e58 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:14.550423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3636f5-d230-47e6-b66d-937f06f55ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:14.551066 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3636f5-d230-47e6-b66d-937f06f55ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.551346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3636f5-d230-47e6-b66d-937f06f55ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.551621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3636f5-d230-47e6-b66d-937f06f55ab9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.552203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3636f5-d230-47e6-b66d-937f06f55ab9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:14.553609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3636f5-d230-47e6-b66d-937f06f55ab9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:14.562036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13c147-13c0-493a-8383-354ccf36ceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:14.562646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13c147-13c0-493a-8383-354ccf36ceb5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.562920 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13c147-13c0-493a-8383-354ccf36ceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6f3636f5-d230-47e6-b66d-937f06f55ab9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:14.563125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:14.564085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:14.570229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:14.570985 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13c147-13c0-493a-8383-354ccf36ceb5 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.570985 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13c147-13c0-493a-8383-354ccf36ceb5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:14.570985 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13c147-13c0-493a-8383-354ccf36ceb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:14.571178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 320/644] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:14.571245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13c147-13c0-493a-8383-354ccf36ceb5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:14.571320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff05d4-12eb-4f68-b7cc-ee41b06732ac None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.571320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff05d4-12eb-4f68-b7cc-ee41b06732ac None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.571320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff05d4-12eb-4f68-b7cc-ee41b06732ac None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.571320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 321/645] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:14.572894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff05d4-12eb-4f68-b7cc-ee41b06732ac None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:14.588058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 325/646] 10.222.0.30 () {56 vars in 1149 bytes} [Sat Oct 15 12:44:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:14.589930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbddd8-e5d1-440e-9b87-8eb85e3b696e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.590220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbddd8-e5d1-440e-9b87-8eb85e3b696e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.590492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbddd8-e5d1-440e-9b87-8eb85e3b696e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.608635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d5bbddd8-e5d1-440e-9b87-8eb85e3b696e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:14.609836 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d5bbddd8-e5d1-440e-9b87-8eb85e3b696e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:14.619254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:14.669711 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:14.687088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff05d4-12eb-4f68-b7cc-ee41b06732ac None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:14.695778 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 326/647] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:14.695778 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbddd8-e5d1-440e-9b87-8eb85e3b696e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:14.739572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386d7c5-2220-427e-b066-eb32e807cc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:14.740187 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386d7c5-2220-427e-b066-eb32e807cc49 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.740463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386d7c5-2220-427e-b066-eb32e807cc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.740739 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386d7c5-2220-427e-b066-eb32e807cc49 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.741387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386d7c5-2220-427e-b066-eb32e807cc49 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:14.741692 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386d7c5-2220-427e-b066-eb32e807cc49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:14.743853 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386d7c5-2220-427e-b066-eb32e807cc49 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:14.750293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 322/648] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:14.755461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386d7c5-2220-427e-b066-eb32e807cc49 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:14.799673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbddd8-e5d1-440e-9b87-8eb85e3b696e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:14.848068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab824e-7944-4167-8e93-8983d4c92d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:14.849023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab824e-7944-4167-8e93-8983d4c92d03 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.849395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab824e-7944-4167-8e93-8983d4c92d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.849971 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab824e-7944-4167-8e93-8983d4c92d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.864580 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab824e-7944-4167-8e93-8983d4c92d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:14.866247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab824e-7944-4167-8e93-8983d4c92d03 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:14.866367 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386d7c5-2220-427e-b066-eb32e807cc49 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:14.870785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-1cab824e-7944-4167-8e93-8983d4c92d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:14.890152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:14.891112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:14.892012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:14.892012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:14.892012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:14.893004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 327/649] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:14.897532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab824e-7944-4167-8e93-8983d4c92d03 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:14.906604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35da65f-9bde-4567-8ecf-e811a9bcb8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:14.907220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35da65f-9bde-4567-8ecf-e811a9bcb8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:14.907500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35da65f-9bde-4567-8ecf-e811a9bcb8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:14.907778 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35da65f-9bde-4567-8ecf-e811a9bcb8f3 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:14.908510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35da65f-9bde-4567-8ecf-e811a9bcb8f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:14.908845 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35da65f-9bde-4567-8ecf-e811a9bcb8f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:14.910504 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35da65f-9bde-4567-8ecf-e811a9bcb8f3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:14.921577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 323/650] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:15.009059 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35da65f-9bde-4567-8ecf-e811a9bcb8f3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:15.025183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab824e-7944-4167-8e93-8983d4c92d03 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:15.028790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:15.067520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddc045-36bb-4e80-9624-b0b6e00c200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:15.068302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddc045-36bb-4e80-9624-b0b6e00c200e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:15.068709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddc045-36bb-4e80-9624-b0b6e00c200e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:15.069088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddc045-36bb-4e80-9624-b0b6e00c200e None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:15.069894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddc045-36bb-4e80-9624-b0b6e00c200e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:15.070347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddc045-36bb-4e80-9624-b0b6e00c200e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:15.071915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddc045-36bb-4e80-9624-b0b6e00c200e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:15.086227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 328/651] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:15.092988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddc045-36bb-4e80-9624-b0b6e00c200e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:15.149789 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35da65f-9bde-4567-8ecf-e811a9bcb8f3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:15.169339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:15.195873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31d840-3c3b-4c32-8580-959aa6ff7748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:15.195873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31d840-3c3b-4c32-8580-959aa6ff7748 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:15.195873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31d840-3c3b-4c32-8580-959aa6ff7748 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:15.195873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31d840-3c3b-4c32-8580-959aa6ff7748 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:15.196295 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31d840-3c3b-4c32-8580-959aa6ff7748 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:15.196295 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31d840-3c3b-4c32-8580-959aa6ff7748 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:15.220170 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddc045-36bb-4e80-9624-b0b6e00c200e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:15.255364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e3f61-454d-4083-b124-5e56fb18a9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:15.256130 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e3f61-454d-4083-b124-5e56fb18a9df None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:15.256324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e3f61-454d-4083-b124-5e56fb18a9df None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:15.256536 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e3f61-454d-4083-b124-5e56fb18a9df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:15.263582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e3f61-454d-4083-b124-5e56fb18a9df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:15.265042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e3f61-454d-4083-b124-5e56fb18a9df None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2a0e3f61-454d-4083-b124-5e56fb18a9df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:15.277463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:15.278963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:15.279878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:15.279878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:15.279878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:15.279878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:15.279878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:15.279878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:15.279878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:15.279878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:15.279878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 329/652] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:15.280941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 324/653] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:44:15 2022] POST /identity/v3/services => generated 269 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:44:15.289343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e3f61-454d-4083-b124-5e56fb18a9df None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:15.414427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e3f61-454d-4083-b124-5e56fb18a9df None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:15.434082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:15.460366 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df2b62-3dcb-4b68-ad3d-37f109aa984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:15.461357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df2b62-3dcb-4b68-ad3d-37f109aa984c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:15.461865 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df2b62-3dcb-4b68-ad3d-37f109aa984c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:15.462362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df2b62-3dcb-4b68-ad3d-37f109aa984c None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:15.463285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df2b62-3dcb-4b68-ad3d-37f109aa984c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:15.464009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df2b62-3dcb-4b68-ad3d-37f109aa984c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:15.465712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df2b62-3dcb-4b68-ad3d-37f109aa984c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:15.480694 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 330/654] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:15.486867 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31d840-3c3b-4c32-8580-959aa6ff7748 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:15.580908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:15.608797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31d840-3c3b-4c32-8580-959aa6ff7748 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:15.637732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e036cd9-1a92-487a-a96d-a708b17b0853 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:15.638060 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e036cd9-1a92-487a-a96d-a708b17b0853 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:15.638287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e036cd9-1a92-487a-a96d-a708b17b0853 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:15.640945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 331/655] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:15.645088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292022aa-e305-49c4-919d-da5ae871e950 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:15.645409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292022aa-e305-49c4-919d-da5ae871e950 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:15.645681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292022aa-e305-49c4-919d-da5ae871e950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:15.648154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8e07f-ebc9-405b-a61a-cbe1ae94185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:15.649155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8e07f-ebc9-405b-a61a-cbe1ae94185b None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:15.649792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8e07f-ebc9-405b-a61a-cbe1ae94185b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:15.650283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8e07f-ebc9-405b-a61a-cbe1ae94185b None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/16c94590fafa48f58fd94526a913dd66/roles/61f7e2c2d5024361a93b2b9121344d4f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:15.652883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8e07f-ebc9-405b-a61a-cbe1ae94185b None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=16c94590fafa48f58fd94526a913dd66, role_id=61f7e2c2d5024361a93b2b9121344d4f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:15.654562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8e07f-ebc9-405b-a61a-cbe1ae94185b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:15.664675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-292022aa-e305-49c4-919d-da5ae871e950 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:15.665903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-292022aa-e305-49c4-919d-da5ae871e950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:15.717007 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 325/656] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:44:15 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/16c94590fafa48f58fd94526a913dd66/roles/61f7e2c2d5024361a93b2b9121344d4f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:44:15.751928 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 332/657] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:15 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:16.018748 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8e07f-ebc9-405b-a61a-cbe1ae94185b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:16.125773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8e07f-ebc9-405b-a61a-cbe1ae94185b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:16.161434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3f43e-a799-49b3-818c-d9e3a2da41cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:16.162071 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3f43e-a799-49b3-818c-d9e3a2da41cb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:16.162392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3f43e-a799-49b3-818c-d9e3a2da41cb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:16.162667 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3f43e-a799-49b3-818c-d9e3a2da41cb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:16.163246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3f43e-a799-49b3-818c-d9e3a2da41cb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:16.164676 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3f43e-a799-49b3-818c-d9e3a2da41cb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:16.170615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-94b3f43e-a799-49b3-818c-d9e3a2da41cb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:16.178407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:16.179408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:16.180642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 326/658] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:44:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:16.188488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292022aa-e305-49c4-919d-da5ae871e950 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:16.300013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292022aa-e305-49c4-919d-da5ae871e950 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:16.302318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:16.336740 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69cbad-9b71-4b2b-afc8-458ce4ea8a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:16.337737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69cbad-9b71-4b2b-afc8-458ce4ea8a0b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:16.338229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69cbad-9b71-4b2b-afc8-458ce4ea8a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:16.338762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69cbad-9b71-4b2b-afc8-458ce4ea8a0b None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:16.339684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69cbad-9b71-4b2b-afc8-458ce4ea8a0b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:16.340183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69cbad-9b71-4b2b-afc8-458ce4ea8a0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:16.341760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69cbad-9b71-4b2b-afc8-458ce4ea8a0b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:16.350325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 333/659] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:44:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:16.352613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3f43e-a799-49b3-818c-d9e3a2da41cb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:16.461476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3f43e-a799-49b3-818c-d9e3a2da41cb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:16.497126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd09f5-7a33-432c-9b90-c0eae1a408dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:16.497765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd09f5-7a33-432c-9b90-c0eae1a408dd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:16.498045 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd09f5-7a33-432c-9b90-c0eae1a408dd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:16.498322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd09f5-7a33-432c-9b90-c0eae1a408dd None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:16.498952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd09f5-7a33-432c-9b90-c0eae1a408dd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:16.499256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd09f5-7a33-432c-9b90-c0eae1a408dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:16.500625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd09f5-7a33-432c-9b90-c0eae1a408dd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:16.509562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 327/660] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:44:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:17.201014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2deb8-6476-4ffd-872c-4ff2cb98d4a9 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:17.201419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2deb8-6476-4ffd-872c-4ff2cb98d4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:17.201603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2deb8-6476-4ffd-872c-4ff2cb98d4a9 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:17.203234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 334/661] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:17.209352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f86d26-b450-4322-9b3c-731ee488d0db None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:17.209352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f86d26-b450-4322-9b3c-731ee488d0db None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:17.209352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f86d26-b450-4322-9b3c-731ee488d0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:17.228638 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-80f86d26-b450-4322-9b3c-731ee488d0db None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:17.230829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80f86d26-b450-4322-9b3c-731ee488d0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:17.289862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:17.301945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 328/662] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:17 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:17.576009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2deb8-6476-4ffd-872c-4ff2cb98d4a9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:17.622937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef3d78-efdd-4276-bb8b-dbd61be9f913 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:17.623267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef3d78-efdd-4276-bb8b-dbd61be9f913 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:17.623495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef3d78-efdd-4276-bb8b-dbd61be9f913 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:17.624978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 329/663] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:17.631226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca8fa4-05e9-4f87-817e-acce57fdcd26 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:17.631737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca8fa4-05e9-4f87-817e-acce57fdcd26 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:17.632211 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca8fa4-05e9-4f87-817e-acce57fdcd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:17.654784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-61ca8fa4-05e9-4f87-817e-acce57fdcd26 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:17.656299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-61ca8fa4-05e9-4f87-817e-acce57fdcd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:17.689142 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2deb8-6476-4ffd-872c-4ff2cb98d4a9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:17.735360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de580b4b-e498-4403-8c64-cdbb878fa070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:17.735983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de580b4b-e498-4403-8c64-cdbb878fa070 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:17.736266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de580b4b-e498-4403-8c64-cdbb878fa070 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:17.736546 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de580b4b-e498-4403-8c64-cdbb878fa070 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:17.745213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de580b4b-e498-4403-8c64-cdbb878fa070 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:17.745213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de580b4b-e498-4403-8c64-cdbb878fa070 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:17.745788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 330/664] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:17 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:17.860460 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 335/665] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:44:17 2022] POST /identity/v3/services => generated 241 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:18.005431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405afa5-e167-4921-b1f1-bb8236260d8a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.006142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405afa5-e167-4921-b1f1-bb8236260d8a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.006640 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405afa5-e167-4921-b1f1-bb8236260d8a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.007995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de580b4b-e498-4403-8c64-cdbb878fa070 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:18.008925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 331/666] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:18.019998 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba309b9f-62bb-4554-a570-de423e463c89 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.020625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba309b9f-62bb-4554-a570-de423e463c89 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.021647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba309b9f-62bb-4554-a570-de423e463c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.043343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:18.044623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ba309b9f-62bb-4554-a570-de423e463c89 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:18.045870 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ba309b9f-62bb-4554-a570-de423e463c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:18.137999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 332/667] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:18 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:18.142731 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de580b4b-e498-4403-8c64-cdbb878fa070 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:18.181703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63223456-aab6-4649-b2ae-382bc1aa33af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:18.183022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63223456-aab6-4649-b2ae-382bc1aa33af None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.183380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63223456-aab6-4649-b2ae-382bc1aa33af None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.183662 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63223456-aab6-4649-b2ae-382bc1aa33af None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.184249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63223456-aab6-4649-b2ae-382bc1aa33af None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:18.185658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63223456-aab6-4649-b2ae-382bc1aa33af None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-63223456-aab6-4649-b2ae-382bc1aa33af None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:18.195140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:18.196118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:18.197115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 336/668] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 12:44:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:18.206303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba309b9f-62bb-4554-a570-de423e463c89 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:18.258713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:18.313698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba309b9f-62bb-4554-a570-de423e463c89 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:18.352486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c97e-31f3-4eb8-a0ef-c78d9d083706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:18.353220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c97e-31f3-4eb8-a0ef-c78d9d083706 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.353505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c97e-31f3-4eb8-a0ef-c78d9d083706 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.353777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c97e-31f3-4eb8-a0ef-c78d9d083706 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.354636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12c97e-31f3-4eb8-a0ef-c78d9d083706 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:18.354939 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12c97e-31f3-4eb8-a0ef-c78d9d083706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:18.356361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12c97e-31f3-4eb8-a0ef-c78d9d083706 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:18.364996 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 333/669] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:18.370826 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63223456-aab6-4649-b2ae-382bc1aa33af None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:18.444469 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c97e-31f3-4eb8-a0ef-c78d9d083706 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:18.463116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:18.480260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63223456-aab6-4649-b2ae-382bc1aa33af None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:18.519319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7535d85-64a6-4101-bb3a-9684dbbb0ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:18.520582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7535d85-64a6-4101-bb3a-9684dbbb0ffb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.520856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7535d85-64a6-4101-bb3a-9684dbbb0ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.521145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7535d85-64a6-4101-bb3a-9684dbbb0ffb None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.521791 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7535d85-64a6-4101-bb3a-9684dbbb0ffb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:18.522094 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7535d85-64a6-4101-bb3a-9684dbbb0ffb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:18.524058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7535d85-64a6-4101-bb3a-9684dbbb0ffb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:18.537045 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 337/670] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:18.538549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7535d85-64a6-4101-bb3a-9684dbbb0ffb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:18.575873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c97e-31f3-4eb8-a0ef-c78d9d083706 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:18.591872 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:18.617042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0becdb-912d-4479-8174-17e996bcb445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:18.618041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0becdb-912d-4479-8174-17e996bcb445 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.618545 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0becdb-912d-4479-8174-17e996bcb445 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.618969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0becdb-912d-4479-8174-17e996bcb445 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.632708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0becdb-912d-4479-8174-17e996bcb445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7d0becdb-912d-4479-8174-17e996bcb445 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:18.639314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:18.640473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 12:44:18.641364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:18.649100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 334/671] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 12:44:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:18.650201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0becdb-912d-4479-8174-17e996bcb445 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:18.677903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7535d85-64a6-4101-bb3a-9684dbbb0ffb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:18.722104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd13e5-4413-4840-b58c-775efcb75abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:18.723620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd13e5-4413-4840-b58c-775efcb75abb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.724087 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd13e5-4413-4840-b58c-775efcb75abb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.724541 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd13e5-4413-4840-b58c-775efcb75abb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.726062 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd13e5-4413-4840-b58c-775efcb75abb None admin] RBAC: Adding query filter params (interface=public, service_id=04a195da32f54e1e9129b5a1e2c60b3c, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:18.726735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd13e5-4413-4840-b58c-775efcb75abb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:18.728682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd13e5-4413-4840-b58c-775efcb75abb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:18.737132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 338/672] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 12:44:18 2022] GET /identity/v3/endpoints?service_id=04a195da32f54e1e9129b5a1e2c60b3c&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:18.738405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4ca32-8766-433d-a60d-89360ebd068e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.738694 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4ca32-8766-433d-a60d-89360ebd068e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.738970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4ca32-8766-433d-a60d-89360ebd068e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.740287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 339/673] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:18.743826 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4ca32-8766-433d-a60d-89360ebd068e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:18.761254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:18.779161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0becdb-912d-4479-8174-17e996bcb445 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:18.818302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f0161-2e13-45da-bf9f-9ff2c228e98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:18.820040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f0161-2e13-45da-bf9f-9ff2c228e98a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.820524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f0161-2e13-45da-bf9f-9ff2c228e98a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.820941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f0161-2e13-45da-bf9f-9ff2c228e98a None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.821777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f0161-2e13-45da-bf9f-9ff2c228e98a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:18.822227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f0161-2e13-45da-bf9f-9ff2c228e98a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:18.824866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f0161-2e13-45da-bf9f-9ff2c228e98a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:18.838298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 335/674] 10.222.0.30 () {56 vars in 1115 bytes} [Sat Oct 15 12:44:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:18.838298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552b25c-ba83-45c8-8416-39eb2b544ab7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.838298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552b25c-ba83-45c8-8416-39eb2b544ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.838298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552b25c-ba83-45c8-8416-39eb2b544ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.855203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8552b25c-ba83-45c8-8416-39eb2b544ab7 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:18.856386 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8552b25c-ba83-45c8-8416-39eb2b544ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:18.856702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4ca32-8766-433d-a60d-89360ebd068e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:18.887400 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:18.896492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6082b3-ba18-43a1-8d43-faf455bee181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:18.897098 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6082b3-ba18-43a1-8d43-faf455bee181 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:18.897419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6082b3-ba18-43a1-8d43-faf455bee181 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:18.897692 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6082b3-ba18-43a1-8d43-faf455bee181 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:18.898300 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6082b3-ba18-43a1-8d43-faf455bee181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:18.899666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6082b3-ba18-43a1-8d43-faf455bee181 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:18.917218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 340/675] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 12:44:18 2022] GET /identity/v3/services => generated 1258 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 12:44:18.917218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6082b3-ba18-43a1-8d43-faf455bee181 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:18.950734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 336/676] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:18 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:19.018993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:19.043265 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6082b3-ba18-43a1-8d43-faf455bee181 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:19.081745 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038e6b5-37fe-4fd2-bd8e-1e263fc36046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:19.082392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038e6b5-37fe-4fd2-bd8e-1e263fc36046 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:19.082670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038e6b5-37fe-4fd2-bd8e-1e263fc36046 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:19.082943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038e6b5-37fe-4fd2-bd8e-1e263fc36046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:19.095920 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038e6b5-37fe-4fd2-bd8e-1e263fc36046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:19.096863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038e6b5-37fe-4fd2-bd8e-1e263fc36046 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c038e6b5-37fe-4fd2-bd8e-1e263fc36046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:19.111945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:19.112955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:19.113974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:19.113974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:19.113974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:19.113974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 341/677] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:19.118881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552b25c-ba83-45c8-8416-39eb2b544ab7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:19.223597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552b25c-ba83-45c8-8416-39eb2b544ab7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:19.235292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038e6b5-37fe-4fd2-bd8e-1e263fc36046 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:19.269669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089e47c-58b7-4354-8e6a-57addd49e3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:19.270493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089e47c-58b7-4354-8e6a-57addd49e3bb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:19.270828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089e47c-58b7-4354-8e6a-57addd49e3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:19.271362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089e47c-58b7-4354-8e6a-57addd49e3bb None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:19.272433 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089e47c-58b7-4354-8e6a-57addd49e3bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:19.272868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089e47c-58b7-4354-8e6a-57addd49e3bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:19.274398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089e47c-58b7-4354-8e6a-57addd49e3bb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:19.291098 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 337/678] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:19.297729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089e47c-58b7-4354-8e6a-57addd49e3bb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:19.327458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:19.370447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038e6b5-37fe-4fd2-bd8e-1e263fc36046 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:19.406839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089e47c-58b7-4354-8e6a-57addd49e3bb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:19.409080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:19.417211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48807d0-d6ae-4859-ad94-a785e658840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:19.418638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48807d0-d6ae-4859-ad94-a785e658840f None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:19.418921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48807d0-d6ae-4859-ad94-a785e658840f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:19.419191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48807d0-d6ae-4859-ad94-a785e658840f None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:19.420272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48807d0-d6ae-4859-ad94-a785e658840f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:19.422038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48807d0-d6ae-4859-ad94-a785e658840f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:19.445306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b2d90-8a39-4cee-ba54-8f3ef4f57dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:19.446049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b2d90-8a39-4cee-ba54-8f3ef4f57dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:19.446334 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b2d90-8a39-4cee-ba54-8f3ef4f57dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:19.446607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b2d90-8a39-4cee-ba54-8f3ef4f57dc9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:19.455407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b2d90-8a39-4cee-ba54-8f3ef4f57dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:19.456903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b2d90-8a39-4cee-ba54-8f3ef4f57dc9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-648b2d90-8a39-4cee-ba54-8f3ef4f57dc9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:19.471009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:19.471963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 12:44:19.472948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:19.478047 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 338/679] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 12:44:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:19.479525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b2d90-8a39-4cee-ba54-8f3ef4f57dc9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:19.479735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 342/680] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:44:19 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:44:19.593601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:19.605744 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b2d90-8a39-4cee-ba54-8f3ef4f57dc9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:19.652158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121c9da-87a0-46f0-ae9c-c96cdd0ccddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:19.652778 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121c9da-87a0-46f0-ae9c-c96cdd0ccddb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:19.653055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121c9da-87a0-46f0-ae9c-c96cdd0ccddb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:19.653357 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121c9da-87a0-46f0-ae9c-c96cdd0ccddb None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:19.654088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121c9da-87a0-46f0-ae9c-c96cdd0ccddb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:19.654392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121c9da-87a0-46f0-ae9c-c96cdd0ccddb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:19.655825 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121c9da-87a0-46f0-ae9c-c96cdd0ccddb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:19.676160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 339/681] 10.222.0.30 () {56 vars in 1149 bytes} [Sat Oct 15 12:44:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:19.681483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48807d0-d6ae-4859-ad94-a785e658840f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:19.792956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48807d0-d6ae-4859-ad94-a785e658840f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:19.827953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be08c30-5920-4bae-bd34-ceed5a269fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:19.828488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be08c30-5920-4bae-bd34-ceed5a269fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:19.828763 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be08c30-5920-4bae-bd34-ceed5a269fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:19.829030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be08c30-5920-4bae-bd34-ceed5a269fe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:19.891143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:19.902623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be08c30-5920-4bae-bd34-ceed5a269fe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:19.904257 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be08c30-5920-4bae-bd34-ceed5a269fe0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:20.011504 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 343/682] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:44:19 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 15 12:44:20.017241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121c9da-87a0-46f0-ae9c-c96cdd0ccddb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:20.131145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121c9da-87a0-46f0-ae9c-c96cdd0ccddb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:20.166375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b440255-e3ea-43b5-8dfd-5d3201658aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:20.167839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b440255-e3ea-43b5-8dfd-5d3201658aaf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:20.168266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b440255-e3ea-43b5-8dfd-5d3201658aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:20.168532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b440255-e3ea-43b5-8dfd-5d3201658aaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:20.169567 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3009275-7aee-4fc2-8170-bb41768177a8 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:20.169853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3009275-7aee-4fc2-8170-bb41768177a8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:20.170127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3009275-7aee-4fc2-8170-bb41768177a8 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:20.171879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 344/683] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:20.176720 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a1b2a-7985-44b4-9550-146ba4cc6a48 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:20.176997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a1b2a-7985-44b4-9550-146ba4cc6a48 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:20.177352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a1b2a-7985-44b4-9550-146ba4cc6a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:20.178551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:20.183191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b440255-e3ea-43b5-8dfd-5d3201658aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:20.184810 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b440255-e3ea-43b5-8dfd-5d3201658aaf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:20.196344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-417a1b2a-7985-44b4-9550-146ba4cc6a48 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:20.197629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-417a1b2a-7985-44b4-9550-146ba4cc6a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8b440255-e3ea-43b5-8dfd-5d3201658aaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:20.203103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:20.208931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:20.210042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:20.210042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:20.210042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:20.210042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 340/684] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:20.211084 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b440255-e3ea-43b5-8dfd-5d3201658aaf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:20.309795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 345/685] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:20.343475 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b440255-e3ea-43b5-8dfd-5d3201658aaf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:20.379289 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1a4c8-633b-410e-9a55-97e8540ebe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:20.380136 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a4c8-633b-410e-9a55-97e8540ebe1c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:20.380558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a4c8-633b-410e-9a55-97e8540ebe1c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:20.380972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a4c8-633b-410e-9a55-97e8540ebe1c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:20.381858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1a4c8-633b-410e-9a55-97e8540ebe1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:20.382385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1a4c8-633b-410e-9a55-97e8540ebe1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:20.383915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1a4c8-633b-410e-9a55-97e8540ebe1c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:20.396455 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 341/686] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:20.405315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a1b2a-7985-44b4-9550-146ba4cc6a48 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:20.428702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:20.512773 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a1b2a-7985-44b4-9550-146ba4cc6a48 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:20.548717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74c28f-ca83-4c2e-9a0b-90c7ad501559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:20.549706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74c28f-ca83-4c2e-9a0b-90c7ad501559 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:20.550183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74c28f-ca83-4c2e-9a0b-90c7ad501559 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:20.550664 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74c28f-ca83-4c2e-9a0b-90c7ad501559 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:20.557654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74c28f-ca83-4c2e-9a0b-90c7ad501559 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:20.561220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74c28f-ca83-4c2e-9a0b-90c7ad501559 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2d74c28f-ca83-4c2e-9a0b-90c7ad501559 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:20.570617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:20.571557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:20.572476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:20.572476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:20.572476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:20.572476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:20.572476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:20.572476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:20.572476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:20.572476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:20.572476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 346/687] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:20.577667 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1a4c8-633b-410e-9a55-97e8540ebe1c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:20.605045 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74c28f-ca83-4c2e-9a0b-90c7ad501559 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:20.631946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:20.692674 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1a4c8-633b-410e-9a55-97e8540ebe1c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:20.706015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:20.728509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74c28f-ca83-4c2e-9a0b-90c7ad501559 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:20.736488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b27709-1a85-4766-b6b0-f37338f92d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:20.737286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b27709-1a85-4766-b6b0-f37338f92d22 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:20.737671 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b27709-1a85-4766-b6b0-f37338f92d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:20.738050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b27709-1a85-4766-b6b0-f37338f92d22 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:20.738843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b27709-1a85-4766-b6b0-f37338f92d22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:20.739254 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b27709-1a85-4766-b6b0-f37338f92d22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:20.741264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b27709-1a85-4766-b6b0-f37338f92d22 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:20.759997 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 342/688] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:20.770141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b27709-1a85-4766-b6b0-f37338f92d22 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:20.775447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabedb1-cc29-4a53-8b3d-ee9304b13215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:20.776694 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabedb1-cc29-4a53-8b3d-ee9304b13215 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:20.776968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabedb1-cc29-4a53-8b3d-ee9304b13215 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:20.777262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabedb1-cc29-4a53-8b3d-ee9304b13215 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:20.777848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabedb1-cc29-4a53-8b3d-ee9304b13215 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:20.779215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabedb1-cc29-4a53-8b3d-ee9304b13215 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-3cabedb1-cc29-4a53-8b3d-ee9304b13215 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:20.788594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:20.789652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:20.791155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 347/689] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:20.797181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabedb1-cc29-4a53-8b3d-ee9304b13215 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:20.894508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b27709-1a85-4766-b6b0-f37338f92d22 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:20.910816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:20.928339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabedb1-cc29-4a53-8b3d-ee9304b13215 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:20.937078 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69299926-efd8-4933-ab19-b8334d8ff92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:20.937807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69299926-efd8-4933-ab19-b8334d8ff92a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:20.938162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69299926-efd8-4933-ab19-b8334d8ff92a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:20.938510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69299926-efd8-4933-ab19-b8334d8ff92a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:20.939428 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69299926-efd8-4933-ab19-b8334d8ff92a None admin] RBAC: Adding query filter params (role.id=61f7e2c2d5024361a93b2b9121344d4f, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=16c94590fafa48f58fd94526a913dd66) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:20.939815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69299926-efd8-4933-ab19-b8334d8ff92a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:20.941482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69299926-efd8-4933-ab19-b8334d8ff92a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:20.950158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 343/690] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:44:20 2022] GET /identity/v3/role_assignments?user.id=16c94590fafa48f58fd94526a913dd66&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=61f7e2c2d5024361a93b2b9121344d4f => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:20.974154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b529958-8c3e-4db9-aba6-41a05c488dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:20.975108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b529958-8c3e-4db9-aba6-41a05c488dcd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:20.975715 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b529958-8c3e-4db9-aba6-41a05c488dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:20.976171 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b529958-8c3e-4db9-aba6-41a05c488dcd None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:20.977037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b529958-8c3e-4db9-aba6-41a05c488dcd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:20.977550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b529958-8c3e-4db9-aba6-41a05c488dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:20.980218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b529958-8c3e-4db9-aba6-41a05c488dcd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:20.986109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 348/691] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:20.996703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69299926-efd8-4933-ab19-b8334d8ff92a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:21.120635 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69299926-efd8-4933-ab19-b8334d8ff92a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:21.159523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06cf60-ee71-4ab7-9c29-8277ac888eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:21.160758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06cf60-ee71-4ab7-9c29-8277ac888eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:21.161297 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06cf60-ee71-4ab7-9c29-8277ac888eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:21.161775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06cf60-ee71-4ab7-9c29-8277ac888eb9 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:21.162632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06cf60-ee71-4ab7-9c29-8277ac888eb9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:21.163152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06cf60-ee71-4ab7-9c29-8277ac888eb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:21.164715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06cf60-ee71-4ab7-9c29-8277ac888eb9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:21.170523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 344/692] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:21.176041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b529958-8c3e-4db9-aba6-41a05c488dcd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:21.189303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:21.287692 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b529958-8c3e-4db9-aba6-41a05c488dcd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:21.324492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8765d96-0617-44ff-bd55-d26002e6dde5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:21.324837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8765d96-0617-44ff-bd55-d26002e6dde5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:21.325055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8765d96-0617-44ff-bd55-d26002e6dde5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:21.326630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 345/693] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:21.327775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8024f7-b527-4318-b2df-4bd97b11b0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:21.328447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8024f7-b527-4318-b2df-4bd97b11b0ae None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:21.328723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8024f7-b527-4318-b2df-4bd97b11b0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:21.328993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8024f7-b527-4318-b2df-4bd97b11b0ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:21.329732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8024f7-b527-4318-b2df-4bd97b11b0ae None admin] RBAC: Adding query filter params (interface=public, service_id=3b6f4ac80d84401889244b4f03ec395d, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:21.330041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8024f7-b527-4318-b2df-4bd97b11b0ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:21.331430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8024f7-b527-4318-b2df-4bd97b11b0ae None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:21.333729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23066e-252b-40af-a194-36a115ba659e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:21.334307 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23066e-252b-40af-a194-36a115ba659e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:21.334836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23066e-252b-40af-a194-36a115ba659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:21.337387 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 349/694] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 12:44:21 2022] GET /identity/v3/endpoints?service_id=3b6f4ac80d84401889244b4f03ec395d&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:21.347113 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8024f7-b527-4318-b2df-4bd97b11b0ae None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:21.361751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ab23066e-252b-40af-a194-36a115ba659e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:21.362982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ab23066e-252b-40af-a194-36a115ba659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:21.405069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:21.431868 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:21.460502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 346/695] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:21.468023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8024f7-b527-4318-b2df-4bd97b11b0ae None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:21.502656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78074d09-8848-4fd1-b636-0493389c43c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:21.503273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78074d09-8848-4fd1-b636-0493389c43c0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:21.503552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78074d09-8848-4fd1-b636-0493389c43c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:21.503828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78074d09-8848-4fd1-b636-0493389c43c0 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:21.504433 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78074d09-8848-4fd1-b636-0493389c43c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:21.505827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78074d09-8848-4fd1-b636-0493389c43c0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:21.512493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 350/696] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 12:44:21 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 12:44:21.737103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23066e-252b-40af-a194-36a115ba659e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:21.825161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f92f6-571b-4168-b026-ccbc9aada055 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:21.825684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f92f6-571b-4168-b026-ccbc9aada055 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:21.825760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f92f6-571b-4168-b026-ccbc9aada055 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:21.828488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 351/697] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:21.832468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743b0b-6dc8-47d3-a0a5-487526a3b155 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:21.832753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743b0b-6dc8-47d3-a0a5-487526a3b155 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:21.833030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743b0b-6dc8-47d3-a0a5-487526a3b155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:21.842003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23066e-252b-40af-a194-36a115ba659e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:21.853787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0d743b0b-6dc8-47d3-a0a5-487526a3b155 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:21.855319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0d743b0b-6dc8-47d3-a0a5-487526a3b155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:21.886847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580164f-ee1b-4bb0-82a2-6a8fcbeecbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:21.887978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580164f-ee1b-4bb0-82a2-6a8fcbeecbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:21.888532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580164f-ee1b-4bb0-82a2-6a8fcbeecbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:21.888816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580164f-ee1b-4bb0-82a2-6a8fcbeecbd0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:21.889529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580164f-ee1b-4bb0-82a2-6a8fcbeecbd0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:21.891978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580164f-ee1b-4bb0-82a2-6a8fcbeecbd0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8580164f-ee1b-4bb0-82a2-6a8fcbeecbd0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:21.905416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:21.906421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:21.907302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 347/698] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 12:44:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:21.913237 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580164f-ee1b-4bb0-82a2-6a8fcbeecbd0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:21.941629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 352/699] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:21.956715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:22.032876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580164f-ee1b-4bb0-82a2-6a8fcbeecbd0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:22.067421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a77f70-ba95-40d3-a5f1-d028a3e6f513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:22.068085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a77f70-ba95-40d3-a5f1-d028a3e6f513 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:22.068365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a77f70-ba95-40d3-a5f1-d028a3e6f513 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:22.068639 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a77f70-ba95-40d3-a5f1-d028a3e6f513 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:22.069340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a77f70-ba95-40d3-a5f1-d028a3e6f513 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:22.069645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a77f70-ba95-40d3-a5f1-d028a3e6f513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:22.071005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a77f70-ba95-40d3-a5f1-d028a3e6f513 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:22.076156 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 348/700] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:22.081577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d743b0b-6dc8-47d3-a0a5-487526a3b155 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:22.152241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:22.188113 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d743b0b-6dc8-47d3-a0a5-487526a3b155 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:22.222879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52a5d9-7893-4e92-9835-1b39f823ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:22.223488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52a5d9-7893-4e92-9835-1b39f823ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:22.223766 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52a5d9-7893-4e92-9835-1b39f823ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:22.224038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52a5d9-7893-4e92-9835-1b39f823ecd4 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:22.224660 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52a5d9-7893-4e92-9835-1b39f823ecd4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:22.224960 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52a5d9-7893-4e92-9835-1b39f823ecd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:22.226421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52a5d9-7893-4e92-9835-1b39f823ecd4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:22.232417 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 353/701] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:22.237738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a77f70-ba95-40d3-a5f1-d028a3e6f513 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:22.246269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52a5d9-7893-4e92-9835-1b39f823ecd4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:22.356574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52a5d9-7893-4e92-9835-1b39f823ecd4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:22.357846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:22.364634 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a77f70-ba95-40d3-a5f1-d028a3e6f513 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:22.391860 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:22.398459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49088710-303a-4e5e-8b59-fff049044490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:22.399290 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49088710-303a-4e5e-8b59-fff049044490 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:22.399733 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49088710-303a-4e5e-8b59-fff049044490 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:22.400162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49088710-303a-4e5e-8b59-fff049044490 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:22.400969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49088710-303a-4e5e-8b59-fff049044490 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:22.402635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49088710-303a-4e5e-8b59-fff049044490 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:22.409628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5dbdf-a469-4e6f-82c4-2a85d51fcb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:22.410636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5dbdf-a469-4e6f-82c4-2a85d51fcb97 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:22.410915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5dbdf-a469-4e6f-82c4-2a85d51fcb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:22.411192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5dbdf-a469-4e6f-82c4-2a85d51fcb97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:22.411846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5dbdf-a469-4e6f-82c4-2a85d51fcb97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:22.413533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5dbdf-a469-4e6f-82c4-2a85d51fcb97 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-49088710-303a-4e5e-8b59-fff049044490 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:22.414197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:22.415435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 12:44:22.417013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:22.424110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 354/702] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:44:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:22.425147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49088710-303a-4e5e-8b59-fff049044490 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:22.476828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 349/703] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 12:44:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:22.540804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49088710-303a-4e5e-8b59-fff049044490 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:22.587712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bb516-353d-4266-978a-30ba2d755dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:22.588865 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bb516-353d-4266-978a-30ba2d755dcc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:22.589148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bb516-353d-4266-978a-30ba2d755dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:22.589461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bb516-353d-4266-978a-30ba2d755dcc None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:22.590166 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bb516-353d-4266-978a-30ba2d755dcc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:22.590538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bb516-353d-4266-978a-30ba2d755dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:22.592009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bb516-353d-4266-978a-30ba2d755dcc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:22.597496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 355/704] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:44:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:22.603587 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5dbdf-a469-4e6f-82c4-2a85d51fcb97 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:22.718304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5dbdf-a469-4e6f-82c4-2a85d51fcb97 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:22.754230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88236a7-ac06-4c0b-9eb9-4858c0e44c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:22.754851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88236a7-ac06-4c0b-9eb9-4858c0e44c6b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:22.755135 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88236a7-ac06-4c0b-9eb9-4858c0e44c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:22.755412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88236a7-ac06-4c0b-9eb9-4858c0e44c6b None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:22.756048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88236a7-ac06-4c0b-9eb9-4858c0e44c6b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:22.756355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88236a7-ac06-4c0b-9eb9-4858c0e44c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:22.757807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88236a7-ac06-4c0b-9eb9-4858c0e44c6b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:22.766337 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 350/705] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:44:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:22.769450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bb516-353d-4266-978a-30ba2d755dcc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:22.834064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:22.876128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bb516-353d-4266-978a-30ba2d755dcc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:22.912399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607882f3-6b52-4092-997d-4e907eddca54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:22.913073 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607882f3-6b52-4092-997d-4e907eddca54 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:22.913446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607882f3-6b52-4092-997d-4e907eddca54 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:22.913731 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607882f3-6b52-4092-997d-4e907eddca54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:22.914436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607882f3-6b52-4092-997d-4e907eddca54 None admin] RBAC: Adding query filter params (interface=public, service_id=e8cdcfd7592c4fecaf79c74d28a029ee, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:22.914753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607882f3-6b52-4092-997d-4e907eddca54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:22.916133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607882f3-6b52-4092-997d-4e907eddca54 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:22.926413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 356/706] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 12:44:22 2022] GET /identity/v3/endpoints?service_id=e8cdcfd7592c4fecaf79c74d28a029ee&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:22.927754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88236a7-ac06-4c0b-9eb9-4858c0e44c6b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:22.949287 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:23.045715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88236a7-ac06-4c0b-9eb9-4858c0e44c6b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:23.083749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44027b75-15ee-4d99-b220-a74d73d31941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:23.084392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44027b75-15ee-4d99-b220-a74d73d31941 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:23.084676 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44027b75-15ee-4d99-b220-a74d73d31941 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:23.085020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44027b75-15ee-4d99-b220-a74d73d31941 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:23.085690 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44027b75-15ee-4d99-b220-a74d73d31941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:23.087094 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44027b75-15ee-4d99-b220-a74d73d31941 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:23.101367 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 351/707] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 12:44:22 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 12:44:23.201249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f02e-4989-4cdf-b66e-3dcf9cbcd13e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:23.201577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f02e-4989-4cdf-b66e-3dcf9cbcd13e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:23.201811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f02e-4989-4cdf-b66e-3dcf9cbcd13e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:23.203521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 357/708] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:23.211130 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17070783-6a0a-4b2d-9471-bbdf820e0c99 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:23.211564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17070783-6a0a-4b2d-9471-bbdf820e0c99 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:23.211945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17070783-6a0a-4b2d-9471-bbdf820e0c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:23.233027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-17070783-6a0a-4b2d-9471-bbdf820e0c99 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:23.234391 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-17070783-6a0a-4b2d-9471-bbdf820e0c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:23.315526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 352/709] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:23.591969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f02e-4989-4cdf-b66e-3dcf9cbcd13e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:23.707044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f02e-4989-4cdf-b66e-3dcf9cbcd13e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:23.726163 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:23.744679 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3391dfe-66a5-4e4d-9025-8f53ac3c5ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:23.745970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3391dfe-66a5-4e4d-9025-8f53ac3c5ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:23.758528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3391dfe-66a5-4e4d-9025-8f53ac3c5ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:23.758528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3391dfe-66a5-4e4d-9025-8f53ac3c5ac0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:23.758528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3391dfe-66a5-4e4d-9025-8f53ac3c5ac0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:23.758528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3391dfe-66a5-4e4d-9025-8f53ac3c5ac0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:23.758528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a3391dfe-66a5-4e4d-9025-8f53ac3c5ac0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 12:44:23.758528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:23.758528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:23.758528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:23.758528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:23.759115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:23.760004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:23.761671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:23.761671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:23.761671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:23.761671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:23.761671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:23.761671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:23.761671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 12:44:23.761671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:23.761671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 358/710] 10.222.0.30 () {56 vars in 1105 bytes} [Sat Oct 15 12:44:23 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:23.765133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17070783-6a0a-4b2d-9471-bbdf820e0c99 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:23.779203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:23.799652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e0b0f-5a0d-44e4-9210-0176b9cb68eb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:23.799936 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e0b0f-5a0d-44e4-9210-0176b9cb68eb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:23.800292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e0b0f-5a0d-44e4-9210-0176b9cb68eb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:23.801723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 359/711] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:23.806962 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015843fd-4cd6-492a-878e-0feacac44a97 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:23.807312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015843fd-4cd6-492a-878e-0feacac44a97 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:23.807573 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015843fd-4cd6-492a-878e-0feacac44a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:23.826955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-015843fd-4cd6-492a-878e-0feacac44a97 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:23.828246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-015843fd-4cd6-492a-878e-0feacac44a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:23.882943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17070783-6a0a-4b2d-9471-bbdf820e0c99 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:23.913867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 360/712] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:23.925963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd6aa4-a41f-4ba6-b70c-5139a8b06e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:23.926593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd6aa4-a41f-4ba6-b70c-5139a8b06e5e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:23.926873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd6aa4-a41f-4ba6-b70c-5139a8b06e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:23.927150 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd6aa4-a41f-4ba6-b70c-5139a8b06e5e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:23.927784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd6aa4-a41f-4ba6-b70c-5139a8b06e5e None admin] RBAC: Adding query filter params (name=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:23.928091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd6aa4-a41f-4ba6-b70c-5139a8b06e5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:23.929485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd6aa4-a41f-4ba6-b70c-5139a8b06e5e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:23.935172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 353/713] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:23.946268 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015843fd-4cd6-492a-878e-0feacac44a97 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:24.054527 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015843fd-4cd6-492a-878e-0feacac44a97 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:24.068004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:24.090002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119559de-ac94-4d40-be98-1c426680f242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:24.090624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119559de-ac94-4d40-be98-1c426680f242 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:24.090907 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119559de-ac94-4d40-be98-1c426680f242 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:24.091182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119559de-ac94-4d40-be98-1c426680f242 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:24.091808 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119559de-ac94-4d40-be98-1c426680f242 None admin] RBAC: Adding query filter params (type=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:24.092112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119559de-ac94-4d40-be98-1c426680f242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:24.093505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119559de-ac94-4d40-be98-1c426680f242 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:24.098248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 361/714] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:23 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:24.189975 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd6aa4-a41f-4ba6-b70c-5139a8b06e5e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:24.273332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:24.299862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd6aa4-a41f-4ba6-b70c-5139a8b06e5e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:24.336242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b37bb-b817-48c3-aed4-30835512c85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:24.336887 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b37bb-b817-48c3-aed4-30835512c85c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:24.337248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b37bb-b817-48c3-aed4-30835512c85c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:24.337534 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b37bb-b817-48c3-aed4-30835512c85c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:24.338125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b37bb-b817-48c3-aed4-30835512c85c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:24.340017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b37bb-b817-48c3-aed4-30835512c85c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-985b37bb-b817-48c3-aed4-30835512c85c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:24.349321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:24.350359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:24.351958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 354/715] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:24.357627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119559de-ac94-4d40-be98-1c426680f242 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:24.465649 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119559de-ac94-4d40-be98-1c426680f242 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:24.501603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692525e-fc8f-4756-bf87-8e76ea124796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:24.502226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692525e-fc8f-4756-bf87-8e76ea124796 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:24.502507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692525e-fc8f-4756-bf87-8e76ea124796 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:24.502783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692525e-fc8f-4756-bf87-8e76ea124796 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:24.503415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692525e-fc8f-4756-bf87-8e76ea124796 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:24.503722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692525e-fc8f-4756-bf87-8e76ea124796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:24.505099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692525e-fc8f-4756-bf87-8e76ea124796 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:24.511040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 362/716] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:24.516392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b37bb-b817-48c3-aed4-30835512c85c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:24.635911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b37bb-b817-48c3-aed4-30835512c85c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:24.672266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f764d82-5ec7-45fa-ae9d-944dd4b349a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:24.673121 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f764d82-5ec7-45fa-ae9d-944dd4b349a9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:24.673570 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f764d82-5ec7-45fa-ae9d-944dd4b349a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:24.674020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f764d82-5ec7-45fa-ae9d-944dd4b349a9 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:24.674795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f764d82-5ec7-45fa-ae9d-944dd4b349a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:24.675244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f764d82-5ec7-45fa-ae9d-944dd4b349a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:24.676740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f764d82-5ec7-45fa-ae9d-944dd4b349a9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:24.682395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 355/717] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:24.688397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692525e-fc8f-4756-bf87-8e76ea124796 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:24.733872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:24.793482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692525e-fc8f-4756-bf87-8e76ea124796 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:24.830378 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda51195-1bba-4504-8ee9-e18bb11e42ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:24.831025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda51195-1bba-4504-8ee9-e18bb11e42ae None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:24.831307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda51195-1bba-4504-8ee9-e18bb11e42ae None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:24.831582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda51195-1bba-4504-8ee9-e18bb11e42ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:24.832272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda51195-1bba-4504-8ee9-e18bb11e42ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:24.833731 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda51195-1bba-4504-8ee9-e18bb11e42ae None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:24.898792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 363/718] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 12:44:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:24.945335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bec4a-faa3-412a-b18e-475e804eb9de None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:24.945886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bec4a-faa3-412a-b18e-475e804eb9de None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:24.945886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bec4a-faa3-412a-b18e-475e804eb9de None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:24.947543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 356/719] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:24.954467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf282ddc-782a-4648-976a-1e32b8335fe9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:24.955000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf282ddc-782a-4648-976a-1e32b8335fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:24.955000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf282ddc-782a-4648-976a-1e32b8335fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:24.975388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-cf282ddc-782a-4648-976a-1e32b8335fe9 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:24.976633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cf282ddc-782a-4648-976a-1e32b8335fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:25.062152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 364/720] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:25.332322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bec4a-faa3-412a-b18e-475e804eb9de None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:25.335213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:25.349380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ffc3f-5737-48af-84fc-39314d014cbf None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:25.349672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ffc3f-5737-48af-84fc-39314d014cbf None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:25.349939 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ffc3f-5737-48af-84fc-39314d014cbf None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:25.351756 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 365/721] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:25.357327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09060d-83d2-4f4f-b149-2dfb66b58525 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:25.357606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09060d-83d2-4f4f-b149-2dfb66b58525 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:25.357881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09060d-83d2-4f4f-b149-2dfb66b58525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:25.377241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8b09060d-83d2-4f4f-b149-2dfb66b58525 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:25.378425 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8b09060d-83d2-4f4f-b149-2dfb66b58525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:25.401246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:25.458557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 366/722] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:25.460530 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bec4a-faa3-412a-b18e-475e804eb9de None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:25.501151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578e3d9-e436-48c3-ba9e-77422e239ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:25.501827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578e3d9-e436-48c3-ba9e-77422e239ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:25.502107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578e3d9-e436-48c3-ba9e-77422e239ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:25.502381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578e3d9-e436-48c3-ba9e-77422e239ce4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:25.502968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578e3d9-e436-48c3-ba9e-77422e239ce4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:25.504332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578e3d9-e436-48c3-ba9e-77422e239ce4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f578e3d9-e436-48c3-ba9e-77422e239ce4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:25.513931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:25.514933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:25.515763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 357/723] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:25.521001 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09060d-83d2-4f4f-b149-2dfb66b58525 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:25.635486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09060d-83d2-4f4f-b149-2dfb66b58525 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:25.670768 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d736b4a-4de7-4742-9172-a27a18303204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:25.671386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d736b4a-4de7-4742-9172-a27a18303204 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:25.671729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d736b4a-4de7-4742-9172-a27a18303204 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:25.672009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d736b4a-4de7-4742-9172-a27a18303204 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:25.672654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d736b4a-4de7-4742-9172-a27a18303204 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:25.672990 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d736b4a-4de7-4742-9172-a27a18303204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:25.674400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d736b4a-4de7-4742-9172-a27a18303204 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:25.679568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 367/724] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:25.685807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578e3d9-e436-48c3-ba9e-77422e239ce4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:25.735383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d736b4a-4de7-4742-9172-a27a18303204 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:25.760286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:25.761656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:25.803238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578e3d9-e436-48c3-ba9e-77422e239ce4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:25.843856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660878-bc0c-4e43-8c30-2d7810f7389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:25.844477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660878-bc0c-4e43-8c30-2d7810f7389e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:25.844758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660878-bc0c-4e43-8c30-2d7810f7389e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:25.845037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660878-bc0c-4e43-8c30-2d7810f7389e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:25.845743 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660878-bc0c-4e43-8c30-2d7810f7389e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:25.846007 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660878-bc0c-4e43-8c30-2d7810f7389e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:25.847383 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660878-bc0c-4e43-8c30-2d7810f7389e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:25.852574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 358/725] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:25.869016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d736b4a-4de7-4742-9172-a27a18303204 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:25.911847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeccb38-fae8-4f5b-a8b7-9afa15f31afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:25.912742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeccb38-fae8-4f5b-a8b7-9afa15f31afc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:25.913165 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeccb38-fae8-4f5b-a8b7-9afa15f31afc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:25.913625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeccb38-fae8-4f5b-a8b7-9afa15f31afc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:25.914394 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeccb38-fae8-4f5b-a8b7-9afa15f31afc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:25.916072 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeccb38-fae8-4f5b-a8b7-9afa15f31afc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4aeccb38-fae8-4f5b-a8b7-9afa15f31afc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:25.927100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:25.928380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 12:44:25.929437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:25.932148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 368/726] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:44:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:25.938467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660878-bc0c-4e43-8c30-2d7810f7389e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:26.057029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660878-bc0c-4e43-8c30-2d7810f7389e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:26.087378 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:26.093833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673e4d4-9c6f-4c46-8f39-05d375da48f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:26.094509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673e4d4-9c6f-4c46-8f39-05d375da48f0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:26.094788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673e4d4-9c6f-4c46-8f39-05d375da48f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:26.095061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673e4d4-9c6f-4c46-8f39-05d375da48f0 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:26.095794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673e4d4-9c6f-4c46-8f39-05d375da48f0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:26.096096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673e4d4-9c6f-4c46-8f39-05d375da48f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:26.097476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673e4d4-9c6f-4c46-8f39-05d375da48f0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:26.102335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 359/727] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:44:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:26.108972 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeccb38-fae8-4f5b-a8b7-9afa15f31afc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:26.196802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:26.217468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeccb38-fae8-4f5b-a8b7-9afa15f31afc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:26.254214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9b8ca-65a8-4867-8cba-e63246ceac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:26.254835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9b8ca-65a8-4867-8cba-e63246ceac39 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:26.255111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9b8ca-65a8-4867-8cba-e63246ceac39 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:26.255384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9b8ca-65a8-4867-8cba-e63246ceac39 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:26.256010 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9b8ca-65a8-4867-8cba-e63246ceac39 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:26.256309 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9b8ca-65a8-4867-8cba-e63246ceac39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:26.257718 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9b8ca-65a8-4867-8cba-e63246ceac39 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:26.264067 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 369/728] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 12:44:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:26.269775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673e4d4-9c6f-4c46-8f39-05d375da48f0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:26.380239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673e4d4-9c6f-4c46-8f39-05d375da48f0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:26.416582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd8968-6c09-41dc-88ea-29e3e40cacd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:26.417438 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd8968-6c09-41dc-88ea-29e3e40cacd1 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:26.417807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd8968-6c09-41dc-88ea-29e3e40cacd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:26.418080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd8968-6c09-41dc-88ea-29e3e40cacd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:26.418757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd8968-6c09-41dc-88ea-29e3e40cacd1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:26.420117 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd8968-6c09-41dc-88ea-29e3e40cacd1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:26.436368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e816f0-600d-4d67-9000-b46726573632 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:26.436658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e816f0-600d-4d67-9000-b46726573632 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:26.441979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e816f0-600d-4d67-9000-b46726573632 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:26.441979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 370/729] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:26.443621 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f77e5-dee2-42ea-a9de-840425cd4144 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:26.443905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f77e5-dee2-42ea-a9de-840425cd4144 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:26.444182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f77e5-dee2-42ea-a9de-840425cd4144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:26.464035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-572f77e5-dee2-42ea-a9de-840425cd4144 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:26.465176 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-572f77e5-dee2-42ea-a9de-840425cd4144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:26.475721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 360/730] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 12:44:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:26.569324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 371/731] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:26.839567 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd8968-6c09-41dc-88ea-29e3e40cacd1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:26.892778 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:26.945253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd8968-6c09-41dc-88ea-29e3e40cacd1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:26.981466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e18bc-0811-45d5-9301-eab7fc182821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:26.982414 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e18bc-0811-45d5-9301-eab7fc182821 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:26.982888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e18bc-0811-45d5-9301-eab7fc182821 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:26.983355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e18bc-0811-45d5-9301-eab7fc182821 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:26.984222 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e18bc-0811-45d5-9301-eab7fc182821 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:26.985808 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e18bc-0811-45d5-9301-eab7fc182821 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:27.054490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 361/732] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:44:26 2022] POST /identity/v3/services => generated 242 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:27.219168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484e02d-1125-4d53-a662-e203595f5799 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:27.219665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484e02d-1125-4d53-a662-e203595f5799 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:27.219665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484e02d-1125-4d53-a662-e203595f5799 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:27.221381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 372/733] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:27.227208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc3f89-ad2a-4c4d-99a9-62b8446f7e87 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:27.227699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc3f89-ad2a-4c4d-99a9-62b8446f7e87 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:27.227800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc3f89-ad2a-4c4d-99a9-62b8446f7e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:27.248420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-92dc3f89-ad2a-4c4d-99a9-62b8446f7e87 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:27.249666 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-92dc3f89-ad2a-4c4d-99a9-62b8446f7e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:27.332991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 362/734] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:27.609313 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484e02d-1125-4d53-a662-e203595f5799 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:27.684262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:27.717406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484e02d-1125-4d53-a662-e203595f5799 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:27.752154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d33c2f-171d-45aa-b973-ec6677d717fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:27.752796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d33c2f-171d-45aa-b973-ec6677d717fb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:27.753080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d33c2f-171d-45aa-b973-ec6677d717fb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:27.753390 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d33c2f-171d-45aa-b973-ec6677d717fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:27.753982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d33c2f-171d-45aa-b973-ec6677d717fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:27.755361 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d33c2f-171d-45aa-b973-ec6677d717fb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e8d33c2f-171d-45aa-b973-ec6677d717fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:27.764997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:27.766047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:27.766876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:27.771157 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 373/735] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:27.774296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc3f89-ad2a-4c4d-99a9-62b8446f7e87 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:27.811800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:27.880561 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc3f89-ad2a-4c4d-99a9-62b8446f7e87 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:27.917970 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c52555-1cd8-4933-982f-6ed4383380e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:27.918608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c52555-1cd8-4933-982f-6ed4383380e8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:27.918871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c52555-1cd8-4933-982f-6ed4383380e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:27.919146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c52555-1cd8-4933-982f-6ed4383380e8 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:27.919775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c52555-1cd8-4933-982f-6ed4383380e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:27.920081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c52555-1cd8-4933-982f-6ed4383380e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:27.921489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c52555-1cd8-4933-982f-6ed4383380e8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:27.931197 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 363/736] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:27.933441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d33c2f-171d-45aa-b973-ec6677d717fb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:28.045392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d33c2f-171d-45aa-b973-ec6677d717fb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:28.084004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23afca-d15a-465a-80f2-82c4b951d7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:28.084973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23afca-d15a-465a-80f2-82c4b951d7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:28.085488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23afca-d15a-465a-80f2-82c4b951d7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:28.085954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23afca-d15a-465a-80f2-82c4b951d7f7 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:28.086775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23afca-d15a-465a-80f2-82c4b951d7f7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:28.087273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23afca-d15a-465a-80f2-82c4b951d7f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:28.089453 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23afca-d15a-465a-80f2-82c4b951d7f7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:28.090927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:28.095603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 374/737] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:28.102552 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c52555-1cd8-4933-982f-6ed4383380e8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:28.142181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37261b3f-a180-4b55-9a9d-3c4658de30a6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:28.142844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37261b3f-a180-4b55-9a9d-3c4658de30a6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:28.143355 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37261b3f-a180-4b55-9a9d-3c4658de30a6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:28.145272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 375/738] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:28.150015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94e063-167b-40d7-a2cc-c36141f1f395 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:28.150303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94e063-167b-40d7-a2cc-c36141f1f395 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:28.150877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94e063-167b-40d7-a2cc-c36141f1f395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:28.173085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-db94e063-167b-40d7-a2cc-c36141f1f395 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:28.176170 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db94e063-167b-40d7-a2cc-c36141f1f395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:28.213621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:28.216558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c52555-1cd8-4933-982f-6ed4383380e8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:28.255573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2c790-ce16-4777-9351-cb800ec562e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:28.256836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2c790-ce16-4777-9351-cb800ec562e6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:28.257306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2c790-ce16-4777-9351-cb800ec562e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:28.257691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 376/739] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:28.258208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2c790-ce16-4777-9351-cb800ec562e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:28.259081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2c790-ce16-4777-9351-cb800ec562e6 None admin] RBAC: Adding query filter params (interface=admin, service_id=3b6f4ac80d84401889244b4f03ec395d, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:28.259582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2c790-ce16-4777-9351-cb800ec562e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:28.261211 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2c790-ce16-4777-9351-cb800ec562e6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:28.267116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 364/740] 10.222.0.30 () {56 vars in 1249 bytes} [Sat Oct 15 12:44:28 2022] GET /identity/v3/endpoints?service_id=3b6f4ac80d84401889244b4f03ec395d&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:28.273140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94e063-167b-40d7-a2cc-c36141f1f395 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:28.384863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94e063-167b-40d7-a2cc-c36141f1f395 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:28.414071 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:28.423356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d639be-b2b7-44be-9b0a-cbc424588ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:28.424259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d639be-b2b7-44be-9b0a-cbc424588ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:28.424744 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d639be-b2b7-44be-9b0a-cbc424588ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:28.425246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d639be-b2b7-44be-9b0a-cbc424588ee5 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:28.426055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d639be-b2b7-44be-9b0a-cbc424588ee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:28.427675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d639be-b2b7-44be-9b0a-cbc424588ee5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:28.439323 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 377/741] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 12:44:28 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 12:44:28.531902 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2c790-ce16-4777-9351-cb800ec562e6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:28.650233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2c790-ce16-4777-9351-cb800ec562e6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:28.690855 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c16cca-dddf-48bb-acec-6b570ba9d4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:28.691470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c16cca-dddf-48bb-acec-6b570ba9d4dc None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:28.691747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c16cca-dddf-48bb-acec-6b570ba9d4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:28.692015 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c16cca-dddf-48bb-acec-6b570ba9d4dc None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:28.692670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c16cca-dddf-48bb-acec-6b570ba9d4dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:28.694069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c16cca-dddf-48bb-acec-6b570ba9d4dc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:28.761686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 365/742] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:44:28 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:28.773890 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458abcb9-8110-4562-b4b3-4cd2e7a0d26e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:28.774172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458abcb9-8110-4562-b4b3-4cd2e7a0d26e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:28.774448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458abcb9-8110-4562-b4b3-4cd2e7a0d26e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:28.776962 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 378/743] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:28.783119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a6e4b-481f-4aa3-ad98-d5f32a2a44de None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:28.783398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a6e4b-481f-4aa3-ad98-d5f32a2a44de None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:28.783678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a6e4b-481f-4aa3-ad98-d5f32a2a44de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:28.805067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d81a6e4b-481f-4aa3-ad98-d5f32a2a44de None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:28.806353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d81a6e4b-481f-4aa3-ad98-d5f32a2a44de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:28.828100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:28.897626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 366/744] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:28 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:29.163563 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458abcb9-8110-4562-b4b3-4cd2e7a0d26e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:29.268608 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458abcb9-8110-4562-b4b3-4cd2e7a0d26e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:29.422961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:29.423495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:29.423772 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:29.424042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:29.424679 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:29.426124 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:29.428689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING keystone.server.flask.common [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 15 12:44:29.468530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.sql.core [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 12:44:29.468530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 12:44:29.468530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [parameters: {'id': 'acfb8d4564f14111874bdeaeb80c3743', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 12:44:29.468530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81613) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 12:44:29.472454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdac9a-f6ab-4700-9166-c08daf6e094e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:29.472912 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdac9a-f6ab-4700-9166-c08daf6e094e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:29.472991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdac9a-f6ab-4700-9166-c08daf6e094e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:29.474633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 367/745] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id acfb8d4564f14111874bdeaeb80c3743.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id acfb8d4564f14111874bdeaeb80c3743. Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:44:29.476316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 12:44:29.477106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application yield resource Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise self.value Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application session.commit() Oct 15 12:44:29.478724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.flush() Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application compat.raise_( Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 12:44:29.479786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 12:44:29.480689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:44:29.481508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [parameters: {'id': 'acfb8d4564f14111874bdeaeb80c3743', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:29.482402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:29.483241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id acfb8d4564f14111874bdeaeb80c3743. Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:29.484141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 379/746] 10.222.0.30 () {60 vars in 1140 bytes} [Sat Oct 15 12:44:29 2022] POST /identity/v3/projects => generated 262 bytes in 315 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:29.484703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ada9f6-736a-4bd6-ab6a-c01c4c7cd737 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:29.484703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ada9f6-736a-4bd6-ab6a-c01c4c7cd737 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:29.484703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ada9f6-736a-4bd6-ab6a-c01c4c7cd737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:29.488485 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:29.504110 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d1ada9f6-736a-4bd6-ab6a-c01c4c7cd737 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:29.506090 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d1ada9f6-736a-4bd6-ab6a-c01c4c7cd737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:29.543716 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:29.601246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 368/747] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:29 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:29.609693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62383bc0-65a5-4376-81c7-d6271e288366 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:29.646653 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96658912-f8f6-4339-b522-7fca3733111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:29.647527 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96658912-f8f6-4339-b522-7fca3733111e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:29.647970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96658912-f8f6-4339-b522-7fca3733111e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:29.648381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96658912-f8f6-4339-b522-7fca3733111e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:29.656068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96658912-f8f6-4339-b522-7fca3733111e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:29.657881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96658912-f8f6-4339-b522-7fca3733111e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-96658912-f8f6-4339-b522-7fca3733111e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:29.670109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:29.671190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:29.672316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:29.672316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:29.672316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:29.672316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:29.672316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:29.672316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:29.672316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:29.672316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:29.674335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 380/748] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:29.680229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ada9f6-736a-4bd6-ab6a-c01c4c7cd737 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:29.770028 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:29.789409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ada9f6-736a-4bd6-ab6a-c01c4c7cd737 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:29.826586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb242a7-ca33-4565-867a-f9eb313e17c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:29.827217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb242a7-ca33-4565-867a-f9eb313e17c7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:29.827498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb242a7-ca33-4565-867a-f9eb313e17c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:29.827773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb242a7-ca33-4565-867a-f9eb313e17c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:29.828437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb242a7-ca33-4565-867a-f9eb313e17c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:29.828746 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb242a7-ca33-4565-867a-f9eb313e17c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:29.830211 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb242a7-ca33-4565-867a-f9eb313e17c7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:29.848371 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 369/749] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:29.883207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96658912-f8f6-4339-b522-7fca3733111e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:29.992873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96658912-f8f6-4339-b522-7fca3733111e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:30.003362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:30.030372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1676008-0c34-4677-aeca-9ee87b1ed31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:30.031023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1676008-0c34-4677-aeca-9ee87b1ed31e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:30.031302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1676008-0c34-4677-aeca-9ee87b1ed31e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:30.031584 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1676008-0c34-4677-aeca-9ee87b1ed31e None admin] PATH_INFO: `/v3/services/image` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:30.032165 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1676008-0c34-4677-aeca-9ee87b1ed31e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:30.033585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1676008-0c34-4677-aeca-9ee87b1ed31e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b1676008-0c34-4677-aeca-9ee87b1ed31e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:30.043952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:30.044991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:30.045884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 381/750] 10.222.0.30 () {56 vars in 1105 bytes} [Sat Oct 15 12:44:29 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:30.051511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb242a7-ca33-4565-867a-f9eb313e17c7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:30.165250 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb242a7-ca33-4565-867a-f9eb313e17c7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:30.201087 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594eb35-1dbc-4804-a455-b2321024d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:30.201726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594eb35-1dbc-4804-a455-b2321024d44a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:30.202003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594eb35-1dbc-4804-a455-b2321024d44a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:30.202276 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594eb35-1dbc-4804-a455-b2321024d44a None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:30.202900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594eb35-1dbc-4804-a455-b2321024d44a None admin] RBAC: Adding query filter params (name=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:30.203201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594eb35-1dbc-4804-a455-b2321024d44a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:30.204617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594eb35-1dbc-4804-a455-b2321024d44a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:30.210154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 370/751] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:30.216168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1676008-0c34-4677-aeca-9ee87b1ed31e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:30.326379 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1676008-0c34-4677-aeca-9ee87b1ed31e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:30.363786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037d832-4290-4d0a-b863-a5fb187ee3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:30.364809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037d832-4290-4d0a-b863-a5fb187ee3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:30.365309 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037d832-4290-4d0a-b863-a5fb187ee3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:30.365774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037d832-4290-4d0a-b863-a5fb187ee3a6 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:30.366607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037d832-4290-4d0a-b863-a5fb187ee3a6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:30.367114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037d832-4290-4d0a-b863-a5fb187ee3a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:30.368669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037d832-4290-4d0a-b863-a5fb187ee3a6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:30.377884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 382/752] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:30.384635 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594eb35-1dbc-4804-a455-b2321024d44a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:30.388282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:30.490157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594eb35-1dbc-4804-a455-b2321024d44a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:30.524936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dad9bc-537d-4f18-a9dd-4e70d008bc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:30.525641 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad9bc-537d-4f18-a9dd-4e70d008bc88 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:30.525915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad9bc-537d-4f18-a9dd-4e70d008bc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:30.526197 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad9bc-537d-4f18-a9dd-4e70d008bc88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:30.527048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dad9bc-537d-4f18-a9dd-4e70d008bc88 None admin] RBAC: Adding query filter params (interface=public, service_id=4201dc50a7ee4445a8c3b46127d072d0, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:30.527354 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dad9bc-537d-4f18-a9dd-4e70d008bc88 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:30.528720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dad9bc-537d-4f18-a9dd-4e70d008bc88 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:30.535437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 371/753] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 12:44:30 2022] GET /identity/v3/endpoints?service_id=4201dc50a7ee4445a8c3b46127d072d0&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:30.541040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037d832-4290-4d0a-b863-a5fb187ee3a6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:30.583032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:30.657205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037d832-4290-4d0a-b863-a5fb187ee3a6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:30.695916 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002008b3-67c7-44d8-ae57-b7294836a285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:30.696522 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002008b3-67c7-44d8-ae57-b7294836a285 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:30.696797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002008b3-67c7-44d8-ae57-b7294836a285 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:30.697063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002008b3-67c7-44d8-ae57-b7294836a285 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:30.698065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002008b3-67c7-44d8-ae57-b7294836a285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:30.699227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002008b3-67c7-44d8-ae57-b7294836a285 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:30.706252 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 383/754] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 12:44:30 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 12:44:30.722236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894e2b0-87c5-4945-9eb1-ddba051aaafb None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:30.722519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894e2b0-87c5-4945-9eb1-ddba051aaafb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:30.722804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894e2b0-87c5-4945-9eb1-ddba051aaafb None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:30.724538 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 372/755] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:30.730986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473fce9-8f4b-466b-a931-714bc357e6c6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:30.731271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473fce9-8f4b-466b-a931-714bc357e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:30.731552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473fce9-8f4b-466b-a931-714bc357e6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:30.751833 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1473fce9-8f4b-466b-a931-714bc357e6c6 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:30.753435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1473fce9-8f4b-466b-a931-714bc357e6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:30.830830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 384/756] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:30 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:31.063762 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2839d76-3321-413e-b8d2-46ffc1411b44 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:31.064282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2839d76-3321-413e-b8d2-46ffc1411b44 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:31.064704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2839d76-3321-413e-b8d2-46ffc1411b44 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:31.066456 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 373/757] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:31.071957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b03ec-9b1e-4c00-9dc9-554c05a9841b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:31.072957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b03ec-9b1e-4c00-9dc9-554c05a9841b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:31.073482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b03ec-9b1e-4c00-9dc9-554c05a9841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:31.092776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dd2b03ec-9b1e-4c00-9dc9-554c05a9841b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:31.094374 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dd2b03ec-9b1e-4c00-9dc9-554c05a9841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:31.109729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2839d76-3321-413e-b8d2-46ffc1411b44 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:31.118832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:31.181229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 385/758] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:31 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:31.197700 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:31.232278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2839d76-3321-413e-b8d2-46ffc1411b44 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:31.268112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161762a4-8d72-43b9-99b1-1c407e693820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:31.268747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161762a4-8d72-43b9-99b1-1c407e693820 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:31.269031 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161762a4-8d72-43b9-99b1-1c407e693820 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:31.269344 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161762a4-8d72-43b9-99b1-1c407e693820 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:31.269933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161762a4-8d72-43b9-99b1-1c407e693820 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:31.271415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161762a4-8d72-43b9-99b1-1c407e693820 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-161762a4-8d72-43b9-99b1-1c407e693820 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:31.280875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:31.281896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 12:44:31.282736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:31.287277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 374/759] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 12:44:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:31.290288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b03ec-9b1e-4c00-9dc9-554c05a9841b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:31.392439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b03ec-9b1e-4c00-9dc9-554c05a9841b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:31.428789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca30fdb-0ace-453f-b833-78cc2a632c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:31.429780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca30fdb-0ace-453f-b833-78cc2a632c70 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:31.430268 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca30fdb-0ace-453f-b833-78cc2a632c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:31.430723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca30fdb-0ace-453f-b833-78cc2a632c70 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:31.431566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca30fdb-0ace-453f-b833-78cc2a632c70 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:31.432219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca30fdb-0ace-453f-b833-78cc2a632c70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:31.433798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca30fdb-0ace-453f-b833-78cc2a632c70 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:31.443683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 386/760] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:31.445376 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161762a4-8d72-43b9-99b1-1c407e693820 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:31.472060 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca30fdb-0ace-453f-b833-78cc2a632c70 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:31.499184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:31.580457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161762a4-8d72-43b9-99b1-1c407e693820 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:31.594863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca30fdb-0ace-453f-b833-78cc2a632c70 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:31.610522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:31.616631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f1691-41e9-4453-ad0f-9834f39f9982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:31.617514 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f1691-41e9-4453-ad0f-9834f39f9982 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:31.617793 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f1691-41e9-4453-ad0f-9834f39f9982 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:31.629043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f1691-41e9-4453-ad0f-9834f39f9982 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:31.629043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f1691-41e9-4453-ad0f-9834f39f9982 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:31.629043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f1691-41e9-4453-ad0f-9834f39f9982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:31.629043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f1691-41e9-4453-ad0f-9834f39f9982 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:31.632222 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 375/761] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 12:44:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:31.633886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f1691-41e9-4453-ad0f-9834f39f9982 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:31.638825 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b5158-11e2-42f3-9fa0-02d97b227715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:31.640520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5158-11e2-42f3-9fa0-02d97b227715 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:31.640979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5158-11e2-42f3-9fa0-02d97b227715 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:31.641469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5158-11e2-42f3-9fa0-02d97b227715 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:31.642248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b5158-11e2-42f3-9fa0-02d97b227715 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:31.643949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b5158-11e2-42f3-9fa0-02d97b227715 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f80b5158-11e2-42f3-9fa0-02d97b227715 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:31.654635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:31.655668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:31.656512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 387/762] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:31.661743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b5158-11e2-42f3-9fa0-02d97b227715 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:31.749870 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f1691-41e9-4453-ad0f-9834f39f9982 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:31.786405 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc241cd1-f424-4ce5-8ad3-9ff08801a259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:31.787383 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc241cd1-f424-4ce5-8ad3-9ff08801a259 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:31.787861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc241cd1-f424-4ce5-8ad3-9ff08801a259 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:31.788326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc241cd1-f424-4ce5-8ad3-9ff08801a259 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:31.789220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc241cd1-f424-4ce5-8ad3-9ff08801a259 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:31.790775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc241cd1-f424-4ce5-8ad3-9ff08801a259 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:31.792755 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b5158-11e2-42f3-9fa0-02d97b227715 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:31.806653 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:31.827670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a239e20-4273-470f-9ee9-0a05716fceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:31.828278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a239e20-4273-470f-9ee9-0a05716fceb0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:31.828558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a239e20-4273-470f-9ee9-0a05716fceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:31.828831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a239e20-4273-470f-9ee9-0a05716fceb0 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:31.829494 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a239e20-4273-470f-9ee9-0a05716fceb0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:31.829801 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a239e20-4273-470f-9ee9-0a05716fceb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:31.831176 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a239e20-4273-470f-9ee9-0a05716fceb0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:31.837117 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 388/763] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:31.843286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a239e20-4273-470f-9ee9-0a05716fceb0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:31.873675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 376/764] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 12:44:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:44:31.972354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a239e20-4273-470f-9ee9-0a05716fceb0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:32.016409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965084b6-45b1-4f99-87fe-bc33730672b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:32.017025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965084b6-45b1-4f99-87fe-bc33730672b7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:32.017335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965084b6-45b1-4f99-87fe-bc33730672b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:32.017610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965084b6-45b1-4f99-87fe-bc33730672b7 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:32.018383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965084b6-45b1-4f99-87fe-bc33730672b7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:32.018747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965084b6-45b1-4f99-87fe-bc33730672b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:32.020128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965084b6-45b1-4f99-87fe-bc33730672b7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:32.026427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 389/765] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:32.032577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc241cd1-f424-4ce5-8ad3-9ff08801a259 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:32.099875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:32.148273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc241cd1-f424-4ce5-8ad3-9ff08801a259 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:32.185015 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf86017-b26f-4321-b996-a5a403cea55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:32.185663 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf86017-b26f-4321-b996-a5a403cea55e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:32.185941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf86017-b26f-4321-b996-a5a403cea55e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:32.186213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf86017-b26f-4321-b996-a5a403cea55e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:32.186909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf86017-b26f-4321-b996-a5a403cea55e None admin] RBAC: Adding query filter params (interface=public, service_id=72d7ef90defe41a6bd6b2b8a52cba982, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:32.187218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf86017-b26f-4321-b996-a5a403cea55e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:32.188575 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf86017-b26f-4321-b996-a5a403cea55e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:32.194215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 377/766] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 12:44:32 2022] GET /identity/v3/endpoints?service_id=72d7ef90defe41a6bd6b2b8a52cba982&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:32.201015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965084b6-45b1-4f99-87fe-bc33730672b7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:32.218927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:32.309936 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965084b6-45b1-4f99-87fe-bc33730672b7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:32.327083 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72241610-f9e2-4431-8693-eefa60c012c6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:32.327683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72241610-f9e2-4431-8693-eefa60c012c6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:32.328186 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72241610-f9e2-4431-8693-eefa60c012c6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:32.329741 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 378/767] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:32.335859 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5e56c-0aab-4c12-bf56-5e7f11c94691 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:32.336225 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5e56c-0aab-4c12-bf56-5e7f11c94691 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:32.336510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5e56c-0aab-4c12-bf56-5e7f11c94691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:32.351143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b7a39-560a-4b6f-a7ec-820b57464711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:32.351760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b7a39-560a-4b6f-a7ec-820b57464711 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:32.352038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b7a39-560a-4b6f-a7ec-820b57464711 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:32.352307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b7a39-560a-4b6f-a7ec-820b57464711 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:32.352909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b7a39-560a-4b6f-a7ec-820b57464711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:32.354888 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b7a39-560a-4b6f-a7ec-820b57464711 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:32.360129 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3fb5e56c-0aab-4c12-bf56-5e7f11c94691 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:32.361595 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3fb5e56c-0aab-4c12-bf56-5e7f11c94691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:32.366457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 390/768] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 12:44:32 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 12:44:32.441574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 379/769] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:32 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:32.717855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b7a39-560a-4b6f-a7ec-820b57464711 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:32.797751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:32.820875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b7a39-560a-4b6f-a7ec-820b57464711 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:32.858815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9cd45-17a6-4f7c-8ee1-d4729ef3c8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:32.859434 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9cd45-17a6-4f7c-8ee1-d4729ef3c8a3 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:32.859713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9cd45-17a6-4f7c-8ee1-d4729ef3c8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:32.859986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9cd45-17a6-4f7c-8ee1-d4729ef3c8a3 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:32.860626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9cd45-17a6-4f7c-8ee1-d4729ef3c8a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:32.862082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9cd45-17a6-4f7c-8ee1-d4729ef3c8a3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:32.914328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.sql.core [None req-8bf9cd45-17a6-4f7c-8ee1-d4729ef3c8a3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 12:44:32.914328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 15 12:44:32.914328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [parameters: {'user_id': '05d54a73eb8e4f309312f25a74b5af17', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 15 12:44:32.914328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81613) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8bf9cd45-17a6-4f7c-8ee1-d4729ef3c8a3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 15 12:44:32.922943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:44:32.923931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application yield resource Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise self.value Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application session.commit() Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 12:44:32.924881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.flush() Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application compat.raise_( Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 12:44:32.925802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:44:32.926697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:44:32.927544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [parameters: {'user_id': '05d54a73eb8e4f309312f25a74b5af17', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:32.928381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 15 12:44:32.929689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 15 12:44:32.931211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 15 12:44:32.931211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 15 12:44:32.931211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 12:44:32.931211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 12:44:32.931211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 15 12:44:32.931211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:32.931211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 391/770] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 12:44:32 2022] POST /identity/v3/users => generated 160 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:32.932620 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5e56c-0aab-4c12-bf56-5e7f11c94691 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:32.990454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:33.022533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce57cd-61ec-4330-9275-de3c687adbca None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:33.022879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce57cd-61ec-4330-9275-de3c687adbca None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:33.023157 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce57cd-61ec-4330-9275-de3c687adbca None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:33.033468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 392/771] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:33.035413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008176b9-83db-45bf-92ef-3a6617a7299e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:33.035695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008176b9-83db-45bf-92ef-3a6617a7299e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:33.035970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008176b9-83db-45bf-92ef-3a6617a7299e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:33.044436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5e56c-0aab-4c12-bf56-5e7f11c94691 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:33.055524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-008176b9-83db-45bf-92ef-3a6617a7299e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:33.057772 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-008176b9-83db-45bf-92ef-3a6617a7299e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:33.091878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4387181-daf4-4552-a83c-f65a7a3c3b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:33.092626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4387181-daf4-4552-a83c-f65a7a3c3b67 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:33.092980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4387181-daf4-4552-a83c-f65a7a3c3b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:33.093355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4387181-daf4-4552-a83c-f65a7a3c3b67 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:33.102509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4387181-daf4-4552-a83c-f65a7a3c3b67 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:33.105228 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4387181-daf4-4552-a83c-f65a7a3c3b67 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d4387181-daf4-4552-a83c-f65a7a3c3b67 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:33.119072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:33.121205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 12:44:33.122387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:33.125267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 380/772] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:44:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:33.128238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4387181-daf4-4552-a83c-f65a7a3c3b67 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:33.147014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 393/773] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:33.242270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4387181-daf4-4552-a83c-f65a7a3c3b67 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:33.253627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:33.278992 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6733-21f0-4a38-87a9-ac6a5aa37936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:33.279687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6733-21f0-4a38-87a9-ac6a5aa37936 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:33.280388 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6733-21f0-4a38-87a9-ac6a5aa37936 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:33.280669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6733-21f0-4a38-87a9-ac6a5aa37936 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:33.281496 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e6733-21f0-4a38-87a9-ac6a5aa37936 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:33.285724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e6733-21f0-4a38-87a9-ac6a5aa37936 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:33.285724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e6733-21f0-4a38-87a9-ac6a5aa37936 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:33.322702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 381/774] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 12:44:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:33.427489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008176b9-83db-45bf-92ef-3a6617a7299e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:33.503862 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:33.538064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008176b9-83db-45bf-92ef-3a6617a7299e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:33.574868 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdebea7-8dd8-4932-9a7d-96e8bca118bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:33.575513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdebea7-8dd8-4932-9a7d-96e8bca118bc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:33.575792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdebea7-8dd8-4932-9a7d-96e8bca118bc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:33.576065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdebea7-8dd8-4932-9a7d-96e8bca118bc None admin] PATH_INFO: `/v3/services/image` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:33.576698 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdebea7-8dd8-4932-9a7d-96e8bca118bc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:33.578241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdebea7-8dd8-4932-9a7d-96e8bca118bc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8bdebea7-8dd8-4932-9a7d-96e8bca118bc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:33.587214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:33.588777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 12:44:33.589634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:33.591911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 394/775] 10.222.0.30 () {56 vars in 1105 bytes} [Sat Oct 15 12:44:33 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:33.595440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6733-21f0-4a38-87a9-ac6a5aa37936 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:33.702495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6733-21f0-4a38-87a9-ac6a5aa37936 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:33.736581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c62d21-0314-4975-912f-03725090248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:33.737199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c62d21-0314-4975-912f-03725090248a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:33.737487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c62d21-0314-4975-912f-03725090248a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:33.737758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c62d21-0314-4975-912f-03725090248a None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:33.738379 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c62d21-0314-4975-912f-03725090248a None admin] RBAC: Adding query filter params (name=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:33.738680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c62d21-0314-4975-912f-03725090248a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:33.740040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c62d21-0314-4975-912f-03725090248a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:33.745543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 382/776] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:33.750946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdebea7-8dd8-4932-9a7d-96e8bca118bc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:33.850717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdebea7-8dd8-4932-9a7d-96e8bca118bc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:33.879693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:33.885973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ed9f6-aa20-4493-9878-b4ee2572f3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:33.889659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ed9f6-aa20-4493-9878-b4ee2572f3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:33.889659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ed9f6-aa20-4493-9878-b4ee2572f3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:33.889659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ed9f6-aa20-4493-9878-b4ee2572f3b4 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:33.889659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ed9f6-aa20-4493-9878-b4ee2572f3b4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:33.889659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ed9f6-aa20-4493-9878-b4ee2572f3b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:33.889659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ed9f6-aa20-4493-9878-b4ee2572f3b4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:33.894421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 395/777] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:33.899905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c62d21-0314-4975-912f-03725090248a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:33.933061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:34.002701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c62d21-0314-4975-912f-03725090248a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:34.036976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3166-0ac4-4e2f-b247-4c0893b50d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:34.037666 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3166-0ac4-4e2f-b247-4c0893b50d59 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:34.037945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3166-0ac4-4e2f-b247-4c0893b50d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:34.038217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3166-0ac4-4e2f-b247-4c0893b50d59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:34.038942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d3166-0ac4-4e2f-b247-4c0893b50d59 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:34.040299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d3166-0ac4-4e2f-b247-4c0893b50d59 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:34.141311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 383/778] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 12:44:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:34.268510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40192691-e8b1-4bae-83f0-8f71011ced5d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:34.268842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40192691-e8b1-4bae-83f0-8f71011ced5d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:34.269067 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40192691-e8b1-4bae-83f0-8f71011ced5d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:34.270633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 396/779] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:34.276313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacf7f0-ec43-4a8b-a038-eccbf98d8f3a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:34.276738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacf7f0-ec43-4a8b-a038-eccbf98d8f3a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:34.276949 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacf7f0-ec43-4a8b-a038-eccbf98d8f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:34.297805 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-caacf7f0-ec43-4a8b-a038-eccbf98d8f3a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:34.299054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-caacf7f0-ec43-4a8b-a038-eccbf98d8f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:34.377685 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 384/780] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:34 2022] POST /identity/v3/auth/tokens => generated 1962 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:34.569894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd171f-51fa-4af4-a06a-439bb4c2a387 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:34.570181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd171f-51fa-4af4-a06a-439bb4c2a387 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:34.570470 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd171f-51fa-4af4-a06a-439bb4c2a387 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:34.571930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 397/781] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:34.577731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12c51-e787-49b8-8207-03ba60eefe43 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:34.578020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12c51-e787-49b8-8207-03ba60eefe43 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:34.578293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12c51-e787-49b8-8207-03ba60eefe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:34.596349 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-93e12c51-e787-49b8-8207-03ba60eefe43 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:34.597562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-93e12c51-e787-49b8-8207-03ba60eefe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:34.606995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:34.644037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd171f-51fa-4af4-a06a-439bb4c2a387 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:34.673290 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 385/782] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:34 2022] POST /identity/v3/auth/tokens => generated 1962 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:34.750392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd171f-51fa-4af4-a06a-439bb4c2a387 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:34.783838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6abb3b-ab8f-42af-9a29-cd144db1220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:34.784561 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6abb3b-ab8f-42af-9a29-cd144db1220d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:34.784742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6abb3b-ab8f-42af-9a29-cd144db1220d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:34.785063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6abb3b-ab8f-42af-9a29-cd144db1220d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:34.786446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6abb3b-ab8f-42af-9a29-cd144db1220d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:34.787869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6abb3b-ab8f-42af-9a29-cd144db1220d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:34.794406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 398/783] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:34.795605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12c51-e787-49b8-8207-03ba60eefe43 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:34.897108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12c51-e787-49b8-8207-03ba60eefe43 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:34.930509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679ba82-10ca-4e8d-9eae-778d115be502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:34.931115 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679ba82-10ca-4e8d-9eae-778d115be502 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:34.931392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679ba82-10ca-4e8d-9eae-778d115be502 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:34.932206 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679ba82-10ca-4e8d-9eae-778d115be502 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:34.932854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679ba82-10ca-4e8d-9eae-778d115be502 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:34.934291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679ba82-10ca-4e8d-9eae-778d115be502 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:34.944797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6abb3b-ab8f-42af-9a29-cd144db1220d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:34.954754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:35.008460 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:35.033557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 386/784] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:44:34 2022] POST /identity/v3/projects => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:35.050289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6abb3b-ab8f-42af-9a29-cd144db1220d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:35.088955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70e5b8-83dd-43dd-8260-69439efa3acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:35.090153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70e5b8-83dd-43dd-8260-69439efa3acb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:35.090398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70e5b8-83dd-43dd-8260-69439efa3acb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:35.090678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70e5b8-83dd-43dd-8260-69439efa3acb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:35.091623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70e5b8-83dd-43dd-8260-69439efa3acb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:35.093980 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70e5b8-83dd-43dd-8260-69439efa3acb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-3f70e5b8-83dd-43dd-8260-69439efa3acb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:35.106625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:35.109789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:35.110678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 399/785] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:35.114366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679ba82-10ca-4e8d-9eae-778d115be502 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:35.227856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679ba82-10ca-4e8d-9eae-778d115be502 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:35.264602 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94d912-95f2-4f6c-84a3-0d711054829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:35.265301 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94d912-95f2-4f6c-84a3-0d711054829f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:35.265592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94d912-95f2-4f6c-84a3-0d711054829f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:35.265868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94d912-95f2-4f6c-84a3-0d711054829f None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:35.266505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94d912-95f2-4f6c-84a3-0d711054829f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:35.266813 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94d912-95f2-4f6c-84a3-0d711054829f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:35.268182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94d912-95f2-4f6c-84a3-0d711054829f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:35.273179 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 387/786] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:35.279199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70e5b8-83dd-43dd-8260-69439efa3acb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:35.390671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70e5b8-83dd-43dd-8260-69439efa3acb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:35.420100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:35.429074 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5128f8-4c47-4020-9ade-3bb4ad173e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:35.429978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5128f8-4c47-4020-9ade-3bb4ad173e76 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:35.430475 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5128f8-4c47-4020-9ade-3bb4ad173e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:35.430968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5128f8-4c47-4020-9ade-3bb4ad173e76 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:35.432331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5128f8-4c47-4020-9ade-3bb4ad173e76 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:35.432867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5128f8-4c47-4020-9ade-3bb4ad173e76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:35.434475 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5128f8-4c47-4020-9ade-3bb4ad173e76 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:35.444286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 400/787] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 12:44:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:35.445453 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94d912-95f2-4f6c-84a3-0d711054829f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:35.507833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:35.552061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94d912-95f2-4f6c-84a3-0d711054829f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:35.587832 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89caa72e-bd8d-4197-8ecb-135499c89174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:35.589031 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caa72e-bd8d-4197-8ecb-135499c89174 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:35.589351 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caa72e-bd8d-4197-8ecb-135499c89174 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:35.589626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caa72e-bd8d-4197-8ecb-135499c89174 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:35.590299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89caa72e-bd8d-4197-8ecb-135499c89174 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:35.591669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89caa72e-bd8d-4197-8ecb-135499c89174 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:35.681219 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 388/788] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 12:44:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:36.359359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651dc9b-0610-4f3f-a82b-805df32454ae None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:36.359915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651dc9b-0610-4f3f-a82b-805df32454ae None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:36.359915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651dc9b-0610-4f3f-a82b-805df32454ae None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:36.361428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 401/789] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:36.366509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04499-d09e-44a4-8d1e-ebf7f49761cb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:36.366927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04499-d09e-44a4-8d1e-ebf7f49761cb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:36.367120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04499-d09e-44a4-8d1e-ebf7f49761cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:36.387571 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bca04499-d09e-44a4-8d1e-ebf7f49761cb None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:36.388771 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bca04499-d09e-44a4-8d1e-ebf7f49761cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:36.462592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 389/790] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:36.733081 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651dc9b-0610-4f3f-a82b-805df32454ae None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:36.835396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651dc9b-0610-4f3f-a82b-805df32454ae None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:36.870225 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:36.870836 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:36.871122 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:36.871398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:36.872116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:36.873483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:36.913932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.sql.core [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 12:44:36.913932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 12:44:36.913932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [parameters: {'id': 'd67e173fc9e64673816cd4f962950214', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 12:44:36.913932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81613) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:44:36.921947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 12:44:36.922848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application yield resource Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise self.value Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application session.commit() Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 12:44:36.924266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.flush() Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application compat.raise_( Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 12:44:36.925251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 12:44:36.926208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 12:44:36.927021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [parameters: {'id': 'd67e173fc9e64673816cd4f962950214', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 12:44:36.932610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:36.934477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:36.936853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 402/791] 10.222.0.30 () {60 vars in 1137 bytes} [Sat Oct 15 12:44:36 2022] POST /identity/v3/domains => generated 190 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:36.939533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04499-d09e-44a4-8d1e-ebf7f49761cb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:36.978481 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:37.036891 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04499-d09e-44a4-8d1e-ebf7f49761cb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:37.071395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0086d33-f96a-4ecf-b5f5-9115f71d1e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:37.072035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0086d33-f96a-4ecf-b5f5-9115f71d1e79 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:37.072902 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0086d33-f96a-4ecf-b5f5-9115f71d1e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:37.073232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0086d33-f96a-4ecf-b5f5-9115f71d1e79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:37.083119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0086d33-f96a-4ecf-b5f5-9115f71d1e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:37.084556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0086d33-f96a-4ecf-b5f5-9115f71d1e79 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d0086d33-f96a-4ecf-b5f5-9115f71d1e79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:37.098913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:37.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:37.100898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:37.100898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:37.100898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:37.105365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 390/792] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:37.106181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:37.179875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:37.225307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f587c-bb14-4bf9-a7cf-b16578bc66b2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:37.240614 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fe440-9942-4292-8c59-dea2ce1b397c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:37.240900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fe440-9942-4292-8c59-dea2ce1b397c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:37.241184 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fe440-9942-4292-8c59-dea2ce1b397c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:37.242651 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 391/793] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:37.247674 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15f90b-a641-4739-bebf-7d10d576c25c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:37.247959 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15f90b-a641-4739-bebf-7d10d576c25c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:37.248238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15f90b-a641-4739-bebf-7d10d576c25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:37.262756 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a634603-4dff-4233-ab99-e41b137f1c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:37.263368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a634603-4dff-4233-ab99-e41b137f1c6e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:37.263650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a634603-4dff-4233-ab99-e41b137f1c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:37.263921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a634603-4dff-4233-ab99-e41b137f1c6e None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:37.264632 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a634603-4dff-4233-ab99-e41b137f1c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:37.264859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a634603-4dff-4233-ab99-e41b137f1c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:37.266250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a634603-4dff-4233-ab99-e41b137f1c6e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:37.268112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ec15f90b-a641-4739-bebf-7d10d576c25c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:37.269415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ec15f90b-a641-4739-bebf-7d10d576c25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:37.279911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 403/794] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:37.365804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 392/795] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:37.637391 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a634603-4dff-4233-ab99-e41b137f1c6e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:37.732565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a634603-4dff-4233-ab99-e41b137f1c6e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:37.748753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:37.765478 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e3ed8-f566-4935-837e-10b5a0bfaeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:37.768338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e3ed8-f566-4935-837e-10b5a0bfaeed None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:37.768338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e3ed8-f566-4935-837e-10b5a0bfaeed None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:37.768338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e3ed8-f566-4935-837e-10b5a0bfaeed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:37.768338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e3ed8-f566-4935-837e-10b5a0bfaeed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:37.769095 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e3ed8-f566-4935-837e-10b5a0bfaeed None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:37.770654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 404/796] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:37.777033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15f90b-a641-4739-bebf-7d10d576c25c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:37.803891 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:37.879172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15f90b-a641-4739-bebf-7d10d576c25c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:37.911518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763dfa3-c11e-4dee-9728-7531aeedd3e4 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:37.911728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763dfa3-c11e-4dee-9728-7531aeedd3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:37.912005 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763dfa3-c11e-4dee-9728-7531aeedd3e4 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:37.913116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cda89-76b3-4ad6-9f7b-a85f77fb9300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:37.913416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 405/797] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:37.913760 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cda89-76b3-4ad6-9f7b-a85f77fb9300 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:37.914037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cda89-76b3-4ad6-9f7b-a85f77fb9300 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:37.914309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cda89-76b3-4ad6-9f7b-a85f77fb9300 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:37.914964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cda89-76b3-4ad6-9f7b-a85f77fb9300 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:37.916820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cda89-76b3-4ad6-9f7b-a85f77fb9300 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:37.918444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dc975-f79e-4b2d-b321-24548f6a0ded None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:37.918733 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dc975-f79e-4b2d-b321-24548f6a0ded None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:37.919013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dc975-f79e-4b2d-b321-24548f6a0ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:37.937343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3c5dc975-f79e-4b2d-b321-24548f6a0ded None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:37.938532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3c5dc975-f79e-4b2d-b321-24548f6a0ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:37.993239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 393/798] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:44:37 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:44:38.006736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 406/799] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:38.276942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cda89-76b3-4ad6-9f7b-a85f77fb9300 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:38.370852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:38.389644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cda89-76b3-4ad6-9f7b-a85f77fb9300 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:38.424841 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f263cf0-5690-45e2-8ff4-1e3966ad91c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:38.425502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f263cf0-5690-45e2-8ff4-1e3966ad91c8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:38.425783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f263cf0-5690-45e2-8ff4-1e3966ad91c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:38.426057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f263cf0-5690-45e2-8ff4-1e3966ad91c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:38.436628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f263cf0-5690-45e2-8ff4-1e3966ad91c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8f263cf0-5690-45e2-8ff4-1e3966ad91c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:38.441497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:38.442654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:38.444608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 394/800] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 12:44:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:38.448667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5dc975-f79e-4b2d-b321-24548f6a0ded None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:38.578009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5dc975-f79e-4b2d-b321-24548f6a0ded None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:38.595071 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:38.618803 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c92a7-7219-44ea-a14a-65b0a3df8241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:38.619509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c92a7-7219-44ea-a14a-65b0a3df8241 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:38.619797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c92a7-7219-44ea-a14a-65b0a3df8241 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:38.620153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c92a7-7219-44ea-a14a-65b0a3df8241 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:38.620829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c92a7-7219-44ea-a14a-65b0a3df8241 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:38.621136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c92a7-7219-44ea-a14a-65b0a3df8241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:38.622574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c92a7-7219-44ea-a14a-65b0a3df8241 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:38.631298 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 407/801] 10.222.0.30 () {56 vars in 1115 bytes} [Sat Oct 15 12:44:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:38.638742 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f263cf0-5690-45e2-8ff4-1e3966ad91c8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:38.750736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f263cf0-5690-45e2-8ff4-1e3966ad91c8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:38.788850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeb1eb-0e99-4665-9c52-99f1aa0aa420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:38.789974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeb1eb-0e99-4665-9c52-99f1aa0aa420 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:38.790383 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeb1eb-0e99-4665-9c52-99f1aa0aa420 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:38.790660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeb1eb-0e99-4665-9c52-99f1aa0aa420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:38.800214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:38.803562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdeb1eb-0e99-4665-9c52-99f1aa0aa420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:38.805022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdeb1eb-0e99-4665-9c52-99f1aa0aa420 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7cdeb1eb-0e99-4665-9c52-99f1aa0aa420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:38.826549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:38.828525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:38.830734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:38.830734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:38.830734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:38.832690 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 395/802] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:38.834150 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c92a7-7219-44ea-a14a-65b0a3df8241 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:38.948354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c92a7-7219-44ea-a14a-65b0a3df8241 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:38.986677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade54ba7-d50a-4551-ad5a-0f1348f9e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:38.987652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade54ba7-d50a-4551-ad5a-0f1348f9e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:38.988149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade54ba7-d50a-4551-ad5a-0f1348f9e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:38.988426 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade54ba7-d50a-4551-ad5a-0f1348f9e5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:38.989089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade54ba7-d50a-4551-ad5a-0f1348f9e5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:38.989423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade54ba7-d50a-4551-ad5a-0f1348f9e5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:38.991667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade54ba7-d50a-4551-ad5a-0f1348f9e5c5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:39.010644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 408/803] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:39.016832 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeb1eb-0e99-4665-9c52-99f1aa0aa420 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:39.132402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeb1eb-0e99-4665-9c52-99f1aa0aa420 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:39.171669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c3cb6-8072-4cce-96fd-010de600fa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:39.172652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c3cb6-8072-4cce-96fd-010de600fa26 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:39.173106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c3cb6-8072-4cce-96fd-010de600fa26 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:39.173594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c3cb6-8072-4cce-96fd-010de600fa26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:39.183908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c3cb6-8072-4cce-96fd-010de600fa26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:39.186021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c3cb6-8072-4cce-96fd-010de600fa26 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-105c3cb6-8072-4cce-96fd-010de600fa26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:39.200232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:39.201266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:39.202247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 396/804] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 12:44:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:39.207303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade54ba7-d50a-4551-ad5a-0f1348f9e5c5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:39.244961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:39.319061 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade54ba7-d50a-4551-ad5a-0f1348f9e5c5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:39.352851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49d04a-938e-44b3-ac47-6fea93b395e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:39.353520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49d04a-938e-44b3-ac47-6fea93b395e7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:39.353808 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49d04a-938e-44b3-ac47-6fea93b395e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:39.354082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49d04a-938e-44b3-ac47-6fea93b395e7 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:39.354801 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49d04a-938e-44b3-ac47-6fea93b395e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:39.355110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49d04a-938e-44b3-ac47-6fea93b395e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:39.356533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49d04a-938e-44b3-ac47-6fea93b395e7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:39.374200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 409/805] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 12:44:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:39.380134 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c3cb6-8072-4cce-96fd-010de600fa26 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:39.433277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:39.479849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c3cb6-8072-4cce-96fd-010de600fa26 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:39.513139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d564879-a7b9-45f9-9816-421d939ed6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:39.513793 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d564879-a7b9-45f9-9816-421d939ed6ca None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:39.514074 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d564879-a7b9-45f9-9816-421d939ed6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:39.514349 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d564879-a7b9-45f9-9816-421d939ed6ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:39.524765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d564879-a7b9-45f9-9816-421d939ed6ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:39.526360 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d564879-a7b9-45f9-9816-421d939ed6ca None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6d564879-a7b9-45f9-9816-421d939ed6ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:39.543731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:39.544726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:39.545833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:39.545833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:39.545833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:39.545833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 397/806] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:39.550700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49d04a-938e-44b3-ac47-6fea93b395e7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:39.643423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:39.650037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49d04a-938e-44b3-ac47-6fea93b395e7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:39.684117 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d4b14-7d84-476a-9dc6-07d0b5e6ae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:39.684689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d4b14-7d84-476a-9dc6-07d0b5e6ae5c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:39.684968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d4b14-7d84-476a-9dc6-07d0b5e6ae5c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:39.685292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d4b14-7d84-476a-9dc6-07d0b5e6ae5c None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:39.685933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d4b14-7d84-476a-9dc6-07d0b5e6ae5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:39.686239 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d4b14-7d84-476a-9dc6-07d0b5e6ae5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:39.687602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d4b14-7d84-476a-9dc6-07d0b5e6ae5c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:39.699179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 410/807] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:39.704866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d564879-a7b9-45f9-9816-421d939ed6ca None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:39.802182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d564879-a7b9-45f9-9816-421d939ed6ca None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:39.804451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:39.835829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42894267-872c-4fea-be56-e24723e0dde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:39.836458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42894267-872c-4fea-be56-e24723e0dde4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:39.836738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42894267-872c-4fea-be56-e24723e0dde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:39.837027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42894267-872c-4fea-be56-e24723e0dde4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:39.844396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42894267-872c-4fea-be56-e24723e0dde4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:39.845870 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42894267-872c-4fea-be56-e24723e0dde4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-42894267-872c-4fea-be56-e24723e0dde4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:39.857831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:39.858740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:39.859962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:39.859962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:39.859962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:39.859962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:39.859962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:39.859962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:39.859962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:39.859962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:39.859962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 398/808] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:39.864687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d4b14-7d84-476a-9dc6-07d0b5e6ae5c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:39.962493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d4b14-7d84-476a-9dc6-07d0b5e6ae5c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:39.996084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5f952-1552-4453-ad6f-f478eb5c6904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:39.996695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5f952-1552-4453-ad6f-f478eb5c6904 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:39.996970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5f952-1552-4453-ad6f-f478eb5c6904 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:39.997264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5f952-1552-4453-ad6f-f478eb5c6904 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:39.997946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5f952-1552-4453-ad6f-f478eb5c6904 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:39.998246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5f952-1552-4453-ad6f-f478eb5c6904 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:39.999667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5f952-1552-4453-ad6f-f478eb5c6904 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:40.011347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 411/809] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:40.017541 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42894267-872c-4fea-be56-e24723e0dde4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:40.116238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42894267-872c-4fea-be56-e24723e0dde4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:40.148012 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ed04b-2240-4fba-9d1a-1e8c1bbde889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:40.148665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ed04b-2240-4fba-9d1a-1e8c1bbde889 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:40.148942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ed04b-2240-4fba-9d1a-1e8c1bbde889 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:40.149241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ed04b-2240-4fba-9d1a-1e8c1bbde889 None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/44b2b449acaa45f099c8a56ecbf20ca6/roles/61f7e2c2d5024361a93b2b9121344d4f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:40.151840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ed04b-2240-4fba-9d1a-1e8c1bbde889 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=44b2b449acaa45f099c8a56ecbf20ca6, role_id=61f7e2c2d5024361a93b2b9121344d4f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:40.153291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ed04b-2240-4fba-9d1a-1e8c1bbde889 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:40.223480 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 399/810] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:44:40 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/44b2b449acaa45f099c8a56ecbf20ca6/roles/61f7e2c2d5024361a93b2b9121344d4f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:44:40.350197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954d58b-010b-40bc-b8f0-ca32a3d8b3ea None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:40.350500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954d58b-010b-40bc-b8f0-ca32a3d8b3ea None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:40.350777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954d58b-010b-40bc-b8f0-ca32a3d8b3ea None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:40.352149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 412/811] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:40.358713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1d0ca-1921-4232-80d9-8d4b7e0eb306 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:40.359001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1d0ca-1921-4232-80d9-8d4b7e0eb306 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:40.359300 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1d0ca-1921-4232-80d9-8d4b7e0eb306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:40.379110 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-20d1d0ca-1921-4232-80d9-8d4b7e0eb306 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:40.380340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-20d1d0ca-1921-4232-80d9-8d4b7e0eb306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:40.399849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:40.450287 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 400/812] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:40.717809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954d58b-010b-40bc-b8f0-ca32a3d8b3ea None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:40.733627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:40.818538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954d58b-010b-40bc-b8f0-ca32a3d8b3ea None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:40.852473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473a151-e748-4144-8306-8f93fe0855a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:40.853103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473a151-e748-4144-8306-8f93fe0855a0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:40.853982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473a151-e748-4144-8306-8f93fe0855a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:40.854263 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473a151-e748-4144-8306-8f93fe0855a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:40.864042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473a151-e748-4144-8306-8f93fe0855a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-1473a151-e748-4144-8306-8f93fe0855a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:40.868790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:40.870332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:40.871191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 413/813] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:44:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:40.875978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1d0ca-1921-4232-80d9-8d4b7e0eb306 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:40.973467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1d0ca-1921-4232-80d9-8d4b7e0eb306 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:41.006162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5d8d0-cb1e-4c97-9f80-cc2b8e5b1def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:41.006768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5d8d0-cb1e-4c97-9f80-cc2b8e5b1def None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:41.007046 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5d8d0-cb1e-4c97-9f80-cc2b8e5b1def None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:41.007318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5d8d0-cb1e-4c97-9f80-cc2b8e5b1def None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:41.007974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5d8d0-cb1e-4c97-9f80-cc2b8e5b1def None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:41.008279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5d8d0-cb1e-4c97-9f80-cc2b8e5b1def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:41.010302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5d8d0-cb1e-4c97-9f80-cc2b8e5b1def None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:41.018163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 401/814] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:44:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:41.023926 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473a151-e748-4144-8306-8f93fe0855a0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:41.099242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:41.125319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473a151-e748-4144-8306-8f93fe0855a0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:41.157805 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe244c2-eeaa-46b9-a47a-bb5f0e1f2f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:41.158433 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe244c2-eeaa-46b9-a47a-bb5f0e1f2f22 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:41.158714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe244c2-eeaa-46b9-a47a-bb5f0e1f2f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:41.159029 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe244c2-eeaa-46b9-a47a-bb5f0e1f2f22 None admin] PATH_INFO: `/v3/users/568a81fa9ea04fa0b5d809796e597e8c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:41.177128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe244c2-eeaa-46b9-a47a-bb5f0e1f2f22 None admin] RBAC: Authorizing `identity:get_user(user_id=568a81fa9ea04fa0b5d809796e597e8c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:41.178591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe244c2-eeaa-46b9-a47a-bb5f0e1f2f22 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:41.180368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 414/815] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:44:41 2022] GET /identity/v3/users/568a81fa9ea04fa0b5d809796e597e8c => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:41.186728 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5d8d0-cb1e-4c97-9f80-cc2b8e5b1def None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:41.205444 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:41.283712 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5d8d0-cb1e-4c97-9f80-cc2b8e5b1def None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:41.317347 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d002d3e-a858-4937-baa8-c3377b35a654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:41.317974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d002d3e-a858-4937-baa8-c3377b35a654 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:41.318278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d002d3e-a858-4937-baa8-c3377b35a654 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:41.318555 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d002d3e-a858-4937-baa8-c3377b35a654 None admin] PATH_INFO: `/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:41.319675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d002d3e-a858-4937-baa8-c3377b35a654 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9f3bcd4da84ed7945b3406a994aaa7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:41.321092 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d002d3e-a858-4937-baa8-c3377b35a654 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:41.322862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 402/816] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:44:41 2022] GET /identity/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:41.328765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe244c2-eeaa-46b9-a47a-bb5f0e1f2f22 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:41.427924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe244c2-eeaa-46b9-a47a-bb5f0e1f2f22 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:41.455940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:41.460820 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e51771-e74b-4447-acbe-b90b344677f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:41.461517 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e51771-e74b-4447-acbe-b90b344677f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:41.466500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e51771-e74b-4447-acbe-b90b344677f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:41.466500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e51771-e74b-4447-acbe-b90b344677f1 None admin] PATH_INFO: `/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7/users/568a81fa9ea04fa0b5d809796e597e8c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:41.466500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e51771-e74b-4447-acbe-b90b344677f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c9f3bcd4da84ed7945b3406a994aaa7, user_id=568a81fa9ea04fa0b5d809796e597e8c, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:41.466500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e51771-e74b-4447-acbe-b90b344677f1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:41.538347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 415/817] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:44:41 2022] PUT /identity/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7/users/568a81fa9ea04fa0b5d809796e597e8c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:44:42.393107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2c446-fed3-494d-aed3-1f7b405a7c33 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:42.393689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2c446-fed3-494d-aed3-1f7b405a7c33 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:42.393689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2c446-fed3-494d-aed3-1f7b405a7c33 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:42.395030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 403/818] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:42.400921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9183a-0317-4741-acb7-d3f33ec70dbb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:42.401599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9183a-0317-4741-acb7-d3f33ec70dbb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:42.401599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9183a-0317-4741-acb7-d3f33ec70dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:42.419974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a8e9183a-0317-4741-acb7-d3f33ec70dbb None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:42.421148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a8e9183a-0317-4741-acb7-d3f33ec70dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:42.484782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 416/819] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:42.754061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2c446-fed3-494d-aed3-1f7b405a7c33 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:42.838881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:42.854687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2c446-fed3-494d-aed3-1f7b405a7c33 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:42.888702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a6966-2204-43d7-bb80-82418410a1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:42.889361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a6966-2204-43d7-bb80-82418410a1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:42.889641 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a6966-2204-43d7-bb80-82418410a1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:42.889914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a6966-2204-43d7-bb80-82418410a1d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:42.899696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a6966-2204-43d7-bb80-82418410a1d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-fe1a6966-2204-43d7-bb80-82418410a1d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:42.904529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:42.905486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:42.906346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 404/820] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 12:44:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:42.911823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9183a-0317-4741-acb7-d3f33ec70dbb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:43.006690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9183a-0317-4741-acb7-d3f33ec70dbb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:43.028512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:43.040066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e203-f1c2-4c87-b8b1-0c593acba917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:43.040678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e203-f1c2-4c87-b8b1-0c593acba917 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:43.041706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e203-f1c2-4c87-b8b1-0c593acba917 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:43.041986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e203-f1c2-4c87-b8b1-0c593acba917 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:43.042656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6e203-f1c2-4c87-b8b1-0c593acba917 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:43.042966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6e203-f1c2-4c87-b8b1-0c593acba917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:43.044385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6e203-f1c2-4c87-b8b1-0c593acba917 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:43.053399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 417/821] 10.222.0.30 () {56 vars in 1115 bytes} [Sat Oct 15 12:44:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:43.059247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a6966-2204-43d7-bb80-82418410a1d7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:43.156930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a6966-2204-43d7-bb80-82418410a1d7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:43.194842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e184d0e-891f-40c5-86c8-1dc1361fc244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:43.195475 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e184d0e-891f-40c5-86c8-1dc1361fc244 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:43.195750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e184d0e-891f-40c5-86c8-1dc1361fc244 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:43.196019 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e184d0e-891f-40c5-86c8-1dc1361fc244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:43.208028 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e184d0e-891f-40c5-86c8-1dc1361fc244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:43.210800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e184d0e-891f-40c5-86c8-1dc1361fc244 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:43.215147 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4e184d0e-891f-40c5-86c8-1dc1361fc244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:43.227436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:43.228457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:43.229373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:43.229373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:43.229373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:43.229373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 405/822] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:43.234522 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e203-f1c2-4c87-b8b1-0c593acba917 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:43.335523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e203-f1c2-4c87-b8b1-0c593acba917 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:43.368689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67b339-ddcd-4c08-afae-ed2b52831cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:43.369788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b339-ddcd-4c08-afae-ed2b52831cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:43.370074 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b339-ddcd-4c08-afae-ed2b52831cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:43.370349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b339-ddcd-4c08-afae-ed2b52831cd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:43.370987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67b339-ddcd-4c08-afae-ed2b52831cd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:43.371293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67b339-ddcd-4c08-afae-ed2b52831cd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:43.372657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67b339-ddcd-4c08-afae-ed2b52831cd3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:43.384103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 418/823] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:43.390205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e184d0e-891f-40c5-86c8-1dc1361fc244 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:43.489760 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e184d0e-891f-40c5-86c8-1dc1361fc244 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:43.523247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96940824-c4aa-4e36-b620-acddcadac2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:43.523879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96940824-c4aa-4e36-b620-acddcadac2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:43.524162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96940824-c4aa-4e36-b620-acddcadac2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:43.524439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96940824-c4aa-4e36-b620-acddcadac2e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:43.532961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96940824-c4aa-4e36-b620-acddcadac2e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:43.534502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96940824-c4aa-4e36-b620-acddcadac2e7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-96940824-c4aa-4e36-b620-acddcadac2e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:44:43.548888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:43.550123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:43.551075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 406/824] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 12:44:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:43.556321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67b339-ddcd-4c08-afae-ed2b52831cd3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:43.605709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:43.656243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec3871-9bdc-4809-b81f-7f49f7989464 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:43.656528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec3871-9bdc-4809-b81f-7f49f7989464 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:43.656810 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec3871-9bdc-4809-b81f-7f49f7989464 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:43.658188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 407/825] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:44:43.660937 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67b339-ddcd-4c08-afae-ed2b52831cd3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:43.663407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3a506-0d52-4dd9-8257-a1690ac21753 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:43.663695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3a506-0d52-4dd9-8257-a1690ac21753 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:43.664056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3a506-0d52-4dd9-8257-a1690ac21753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:43.683472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c5f3a506-0d52-4dd9-8257-a1690ac21753 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:43.684671 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c5f3a506-0d52-4dd9-8257-a1690ac21753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:43.696104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c598bb-d497-4cd4-ad3c-cc06e16065a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:43.696711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c598bb-d497-4cd4-ad3c-cc06e16065a2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:43.697005 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c598bb-d497-4cd4-ad3c-cc06e16065a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:43.697318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c598bb-d497-4cd4-ad3c-cc06e16065a2 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:43.698034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c598bb-d497-4cd4-ad3c-cc06e16065a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:43.698336 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c598bb-d497-4cd4-ad3c-cc06e16065a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:43.699749 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c598bb-d497-4cd4-ad3c-cc06e16065a2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:43.717008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 419/826] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 12:44:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:43.722769 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c598bb-d497-4cd4-ad3c-cc06e16065a2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:43.736167 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:43.752560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 408/827] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 12:44:43.824108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c598bb-d497-4cd4-ad3c-cc06e16065a2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:43.825652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:43.857221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01135eb-760b-4edb-b7c9-0ee60f78ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:43.857789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01135eb-760b-4edb-b7c9-0ee60f78ce3a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:43.858072 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01135eb-760b-4edb-b7c9-0ee60f78ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:43.858345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01135eb-760b-4edb-b7c9-0ee60f78ce3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:43.922782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01135eb-760b-4edb-b7c9-0ee60f78ce3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:43.924227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01135eb-760b-4edb-b7c9-0ee60f78ce3a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.019251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 420/828] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:44:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 15 12:44:44.025022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3a506-0d52-4dd9-8257-a1690ac21753 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.033209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01135eb-760b-4edb-b7c9-0ee60f78ce3a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.036000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:44.126416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3a506-0d52-4dd9-8257-a1690ac21753 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.132938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01135eb-760b-4edb-b7c9-0ee60f78ce3a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.162370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8874d4-84f4-477e-8b33-a15a539454e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.162999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8874d4-84f4-477e-8b33-a15a539454e1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.163279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8874d4-84f4-477e-8b33-a15a539454e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.163552 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8874d4-84f4-477e-8b33-a15a539454e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.167188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2385479-47c7-4879-b0aa-89903a493782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.167815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2385479-47c7-4879-b0aa-89903a493782 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.168096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2385479-47c7-4879-b0aa-89903a493782 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.168371 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2385479-47c7-4879-b0aa-89903a493782 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.173903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8874d4-84f4-477e-8b33-a15a539454e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:44.175341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8874d4-84f4-477e-8b33-a15a539454e1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.179455 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2385479-47c7-4879-b0aa-89903a493782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d2385479-47c7-4879-b0aa-89903a493782 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:44:44.184589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:44.185579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:44:44.186414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 421/829] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5e8874d4-84f4-477e-8b33-a15a539454e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:44:44.189942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:44.190945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:44:44.191829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:44:44.191829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:44:44.191829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:44.192013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2385479-47c7-4879-b0aa-89903a493782 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.195526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 409/830] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:44.196336 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8874d4-84f4-477e-8b33-a15a539454e1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.235241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:44.253658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:44.292509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2385479-47c7-4879-b0aa-89903a493782 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.295311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8874d4-84f4-477e-8b33-a15a539454e1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.326083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fe50c-de7d-4423-896e-b6f3ad47d6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.326686 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fe50c-de7d-4423-896e-b6f3ad47d6af None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.326962 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fe50c-de7d-4423-896e-b6f3ad47d6af None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.327233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fe50c-de7d-4423-896e-b6f3ad47d6af None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.327882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fe50c-de7d-4423-896e-b6f3ad47d6af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:44.328181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fe50c-de7d-4423-896e-b6f3ad47d6af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:44.328917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab9c64-9b46-4ebb-8bbe-96f7f7d4970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.329556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab9c64-9b46-4ebb-8bbe-96f7f7d4970e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.329655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fe50c-de7d-4423-896e-b6f3ad47d6af None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.329837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab9c64-9b46-4ebb-8bbe-96f7f7d4970e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.330107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab9c64-9b46-4ebb-8bbe-96f7f7d4970e None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.330743 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab9c64-9b46-4ebb-8bbe-96f7f7d4970e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:44.331044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab9c64-9b46-4ebb-8bbe-96f7f7d4970e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:44.332384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab9c64-9b46-4ebb-8bbe-96f7f7d4970e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.337436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 422/831] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:44.344307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fe50c-de7d-4423-896e-b6f3ad47d6af None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.448284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fe50c-de7d-4423-896e-b6f3ad47d6af None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.458995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 410/832] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:44.465617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab9c64-9b46-4ebb-8bbe-96f7f7d4970e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.471906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:44.482882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e036d5d-8ad1-4a74-8ddc-eebab4acfaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.483504 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e036d5d-8ad1-4a74-8ddc-eebab4acfaac None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.483779 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e036d5d-8ad1-4a74-8ddc-eebab4acfaac None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.484047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e036d5d-8ad1-4a74-8ddc-eebab4acfaac None admin] PATH_INFO: `/v3/users/568a81fa9ea04fa0b5d809796e597e8c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.485161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e036d5d-8ad1-4a74-8ddc-eebab4acfaac None admin] RBAC: Authorizing `identity:get_user(user_id=568a81fa9ea04fa0b5d809796e597e8c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:44.486612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e036d5d-8ad1-4a74-8ddc-eebab4acfaac None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.488247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 423/833] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/users/568a81fa9ea04fa0b5d809796e597e8c => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:44.493766 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e036d5d-8ad1-4a74-8ddc-eebab4acfaac None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.565029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:44.569693 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab9c64-9b46-4ebb-8bbe-96f7f7d4970e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.601529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e036d5d-8ad1-4a74-8ddc-eebab4acfaac None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.603669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0224a2-8085-4c24-a783-87a4ee44307c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.604296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0224a2-8085-4c24-a783-87a4ee44307c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.604569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0224a2-8085-4c24-a783-87a4ee44307c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.604837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0224a2-8085-4c24-a783-87a4ee44307c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.611492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0224a2-8085-4c24-a783-87a4ee44307c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:44.612907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0224a2-8085-4c24-a783-87a4ee44307c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-2e0224a2-8085-4c24-a783-87a4ee44307c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:44:44.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:44:44.625523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:44:44.626628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:44:44.626628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:44:44.626628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:44:44.626628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:44:44.626628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:44:44.626628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:44:44.626628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:44:44.626628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:44:44.626628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 411/834] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:44:44.631370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0224a2-8085-4c24-a783-87a4ee44307c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.637444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5e481-b097-4d20-a9ae-33699eac7457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.638056 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5e481-b097-4d20-a9ae-33699eac7457 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.638337 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5e481-b097-4d20-a9ae-33699eac7457 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.638610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5e481-b097-4d20-a9ae-33699eac7457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.696890 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:44.707948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5e481-b097-4d20-a9ae-33699eac7457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:44.707948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5e481-b097-4d20-a9ae-33699eac7457 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.729646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0224a2-8085-4c24-a783-87a4ee44307c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.762781 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24b8a4-18c6-466a-a902-251660187cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.763391 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24b8a4-18c6-466a-a902-251660187cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.763667 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24b8a4-18c6-466a-a902-251660187cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.763946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24b8a4-18c6-466a-a902-251660187cd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.764654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24b8a4-18c6-466a-a902-251660187cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:44.764962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24b8a4-18c6-466a-a902-251660187cd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:44.766418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24b8a4-18c6-466a-a902-251660187cd0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.778020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 412/835] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:44.783987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24b8a4-18c6-466a-a902-251660187cd0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.802171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:44.807298 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 424/836] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 15 12:44:44.813324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5e481-b097-4d20-a9ae-33699eac7457 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:44.880976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24b8a4-18c6-466a-a902-251660187cd0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.901257 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:44.909790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5e481-b097-4d20-a9ae-33699eac7457 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:44.913973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015710d4-bed0-4e1b-a665-a039e6ce23a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.914354 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015710d4-bed0-4e1b-a665-a039e6ce23a0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.914633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015710d4-bed0-4e1b-a665-a039e6ce23a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.914901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015710d4-bed0-4e1b-a665-a039e6ce23a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.915606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015710d4-bed0-4e1b-a665-a039e6ce23a0 None admin] RBAC: Adding query filter params (role.id=61f7e2c2d5024361a93b2b9121344d4f, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=44b2b449acaa45f099c8a56ecbf20ca6) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:44.916449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015710d4-bed0-4e1b-a665-a039e6ce23a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:44.918032 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015710d4-bed0-4e1b-a665-a039e6ce23a0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.923909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 413/837] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/role_assignments?user.id=44b2b449acaa45f099c8a56ecbf20ca6&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=61f7e2c2d5024361a93b2b9121344d4f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:44.945685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3768a84-716a-4322-8d17-882b65777d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:44.946349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3768a84-716a-4322-8d17-882b65777d0f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:44.946633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3768a84-716a-4322-8d17-882b65777d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:44.946911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3768a84-716a-4322-8d17-882b65777d0f None admin] PATH_INFO: `/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:44.948649 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3768a84-716a-4322-8d17-882b65777d0f None admin] RBAC: Authorizing `identity:get_project(project_id=3c9f3bcd4da84ed7945b3406a994aaa7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:44.950173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3768a84-716a-4322-8d17-882b65777d0f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:44.951902 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 425/838] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:44.958437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015710d4-bed0-4e1b-a665-a039e6ce23a0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:45.047119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:45.066559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015710d4-bed0-4e1b-a665-a039e6ce23a0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:45.101735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d57a6f-a5ec-48e9-a5a1-6c2bf47e0e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:45.102357 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d57a6f-a5ec-48e9-a5a1-6c2bf47e0e66 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:45.102659 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d57a6f-a5ec-48e9-a5a1-6c2bf47e0e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:45.102936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d57a6f-a5ec-48e9-a5a1-6c2bf47e0e66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:45.103810 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d57a6f-a5ec-48e9-a5a1-6c2bf47e0e66 None admin] RBAC: Adding query filter params (role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.project.id=3c9f3bcd4da84ed7945b3406a994aaa7, user.id=568a81fa9ea04fa0b5d809796e597e8c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:45.104123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d57a6f-a5ec-48e9-a5a1-6c2bf47e0e66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:45.105588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d57a6f-a5ec-48e9-a5a1-6c2bf47e0e66 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:45.111476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 414/839] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:44:44 2022] GET /identity/v3/role_assignments?user.id=568a81fa9ea04fa0b5d809796e597e8c&scope.project.id=3c9f3bcd4da84ed7945b3406a994aaa7&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:47.299063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87de30-c9fa-47a8-85a7-3c08afabd7ed None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:47.299594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87de30-c9fa-47a8-85a7-3c08afabd7ed None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:47.299594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87de30-c9fa-47a8-85a7-3c08afabd7ed None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:47.300924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 426/840] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:47.306126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726280b-6e17-4c2e-b388-c3a241527c8e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:47.307048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726280b-6e17-4c2e-b388-c3a241527c8e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:47.307243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726280b-6e17-4c2e-b388-c3a241527c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:47.326774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b726280b-6e17-4c2e-b388-c3a241527c8e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:47.327954 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b726280b-6e17-4c2e-b388-c3a241527c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:47.391805 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 415/841] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:47.658919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87de30-c9fa-47a8-85a7-3c08afabd7ed None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:47.756925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87de30-c9fa-47a8-85a7-3c08afabd7ed None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:47.790050 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026448a-82ba-4c3f-9df7-da1159676f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:47.790699 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026448a-82ba-4c3f-9df7-da1159676f36 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:47.791053 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026448a-82ba-4c3f-9df7-da1159676f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:47.791495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026448a-82ba-4c3f-9df7-da1159676f36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:47.792596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026448a-82ba-4c3f-9df7-da1159676f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:47.794046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026448a-82ba-4c3f-9df7-da1159676f36 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:47.795576 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 427/842] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:44:47.801279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726280b-6e17-4c2e-b388-c3a241527c8e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:47.874029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:47.895851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726280b-6e17-4c2e-b388-c3a241527c8e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:47.928992 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf85fd2-32a9-4473-a02e-6ed20f0a6ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:47.929605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf85fd2-32a9-4473-a02e-6ed20f0a6ec0 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:47.929881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf85fd2-32a9-4473-a02e-6ed20f0a6ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:47.930151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf85fd2-32a9-4473-a02e-6ed20f0a6ec0 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:47.930786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf85fd2-32a9-4473-a02e-6ed20f0a6ec0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:47.932279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf85fd2-32a9-4473-a02e-6ed20f0a6ec0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:48.011682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 416/843] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:44:47 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:50.157173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57885e3b-d669-48df-8a97-f3bce6186949 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:50.157719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57885e3b-d669-48df-8a97-f3bce6186949 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:50.157812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57885e3b-d669-48df-8a97-f3bce6186949 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:50.159078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 428/844] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:50.164500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455b9b7-5fbb-4839-b3ac-33e3b39725a1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:50.164925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455b9b7-5fbb-4839-b3ac-33e3b39725a1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:50.165047 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455b9b7-5fbb-4839-b3ac-33e3b39725a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:50.184094 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1455b9b7-5fbb-4839-b3ac-33e3b39725a1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:50.185308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1455b9b7-5fbb-4839-b3ac-33e3b39725a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:50.248939 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 417/845] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:50.518473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57885e3b-d669-48df-8a97-f3bce6186949 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:50.546982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:50.618543 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57885e3b-d669-48df-8a97-f3bce6186949 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:50.651503 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec06cd9-8e6a-4bc2-b62a-fa2d06e3e71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:50.652126 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec06cd9-8e6a-4bc2-b62a-fa2d06e3e71b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:50.652407 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec06cd9-8e6a-4bc2-b62a-fa2d06e3e71b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:50.652682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec06cd9-8e6a-4bc2-b62a-fa2d06e3e71b None admin] PATH_INFO: `/v3/roles/ffd57ebcb27d45ef9b9d642f409cc6d7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:50.654089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec06cd9-8e6a-4bc2-b62a-fa2d06e3e71b None admin] RBAC: Authorizing `identity:get_role(role_id=ffd57ebcb27d45ef9b9d642f409cc6d7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:50.655496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec06cd9-8e6a-4bc2-b62a-fa2d06e3e71b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:50.656800 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 429/846] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:44:50 2022] GET /identity/v3/roles/ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:50.662947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455b9b7-5fbb-4839-b3ac-33e3b39725a1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:50.721878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:50.757478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455b9b7-5fbb-4839-b3ac-33e3b39725a1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:50.791101 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1715e86-6418-4235-b70e-4b0ea22ce467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:50.791724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1715e86-6418-4235-b70e-4b0ea22ce467 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:50.791999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1715e86-6418-4235-b70e-4b0ea22ce467 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:50.792271 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1715e86-6418-4235-b70e-4b0ea22ce467 None admin] PATH_INFO: `/v3/users/50eadd1da25a44a583af15d561a7cc86` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:50.804736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1715e86-6418-4235-b70e-4b0ea22ce467 None admin] RBAC: Authorizing `identity:get_user(user_id=50eadd1da25a44a583af15d561a7cc86)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:50.806231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1715e86-6418-4235-b70e-4b0ea22ce467 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:50.807838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 418/847] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:44:50 2022] GET /identity/v3/users/50eadd1da25a44a583af15d561a7cc86 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:50.813568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec06cd9-8e6a-4bc2-b62a-fa2d06e3e71b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:50.904341 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:50.909926 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec06cd9-8e6a-4bc2-b62a-fa2d06e3e71b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:50.942854 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8f83-90ee-42d8-bb9a-1bd1d74f196e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:50.943469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8f83-90ee-42d8-bb9a-1bd1d74f196e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:50.943750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8f83-90ee-42d8-bb9a-1bd1d74f196e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:50.944022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8f83-90ee-42d8-bb9a-1bd1d74f196e None admin] PATH_INFO: `/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:50.945114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae8f83-90ee-42d8-bb9a-1bd1d74f196e None admin] RBAC: Authorizing `identity:get_project(project_id=3c9f3bcd4da84ed7945b3406a994aaa7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:50.946602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae8f83-90ee-42d8-bb9a-1bd1d74f196e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:50.948835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 430/848] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:44:50 2022] GET /identity/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:50.954897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1715e86-6418-4235-b70e-4b0ea22ce467 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:51.050264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1715e86-6418-4235-b70e-4b0ea22ce467 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:51.069216 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:51.082713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec306a-21aa-422c-badd-efefddd5bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:51.083358 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec306a-21aa-422c-badd-efefddd5bd0c None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:51.088370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec306a-21aa-422c-badd-efefddd5bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:51.088370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec306a-21aa-422c-badd-efefddd5bd0c None admin] PATH_INFO: `/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7/users/50eadd1da25a44a583af15d561a7cc86/roles/ffd57ebcb27d45ef9b9d642f409cc6d7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:51.088370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec306a-21aa-422c-badd-efefddd5bd0c None admin] RBAC: Authorizing `identity:create_grant(project_id=3c9f3bcd4da84ed7945b3406a994aaa7, user_id=50eadd1da25a44a583af15d561a7cc86, role_id=ffd57ebcb27d45ef9b9d642f409cc6d7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:51.088370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec306a-21aa-422c-badd-efefddd5bd0c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:51.142173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 419/849] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:44:50 2022] PUT /identity/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7/users/50eadd1da25a44a583af15d561a7cc86/roles/ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:44:53.258047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3cd96-9fea-461c-ba45-b3f75191295a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:53.258574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3cd96-9fea-461c-ba45-b3f75191295a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:53.258574 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3cd96-9fea-461c-ba45-b3f75191295a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:53.260453 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 431/850] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:53.265879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc780c-d8c2-44c0-aef3-60ec1ef46b4e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:53.266286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc780c-d8c2-44c0-aef3-60ec1ef46b4e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:53.266429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc780c-d8c2-44c0-aef3-60ec1ef46b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:53.284784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-01cc780c-d8c2-44c0-aef3-60ec1ef46b4e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:53.286255 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-01cc780c-d8c2-44c0-aef3-60ec1ef46b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:53.350517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 420/851] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:53.620827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3cd96-9fea-461c-ba45-b3f75191295a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:53.719369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3cd96-9fea-461c-ba45-b3f75191295a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:53.752575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbcf59-8043-4381-9f13-6d3fd489911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:53.753223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbcf59-8043-4381-9f13-6d3fd489911e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:53.753499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbcf59-8043-4381-9f13-6d3fd489911e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:53.753770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbcf59-8043-4381-9f13-6d3fd489911e None admin] PATH_INFO: `/v3/roles/ffd57ebcb27d45ef9b9d642f409cc6d7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:53.755125 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cbcf59-8043-4381-9f13-6d3fd489911e None admin] RBAC: Authorizing `identity:get_role(role_id=ffd57ebcb27d45ef9b9d642f409cc6d7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:53.756487 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cbcf59-8043-4381-9f13-6d3fd489911e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:53.757822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 432/852] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:44:53 2022] GET /identity/v3/roles/ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:53.763713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc780c-d8c2-44c0-aef3-60ec1ef46b4e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:53.859727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc780c-d8c2-44c0-aef3-60ec1ef46b4e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:53.870451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:53.892369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33335338-b89b-4c84-8f57-8257eaf44b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:53.893056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33335338-b89b-4c84-8f57-8257eaf44b52 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:53.893365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33335338-b89b-4c84-8f57-8257eaf44b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:53.893640 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33335338-b89b-4c84-8f57-8257eaf44b52 None admin] PATH_INFO: `/v3/users/50eadd1da25a44a583af15d561a7cc86` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:53.894737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33335338-b89b-4c84-8f57-8257eaf44b52 None admin] RBAC: Authorizing `identity:get_user(user_id=50eadd1da25a44a583af15d561a7cc86)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:53.896218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33335338-b89b-4c84-8f57-8257eaf44b52 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:53.898104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 421/853] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:44:53 2022] GET /identity/v3/users/50eadd1da25a44a583af15d561a7cc86 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:53.903908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbcf59-8043-4381-9f13-6d3fd489911e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:53.935925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:54.001955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbcf59-8043-4381-9f13-6d3fd489911e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:54.033644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fff61-3a6a-45c3-b127-c72a9fe3d8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:54.034236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fff61-3a6a-45c3-b127-c72a9fe3d8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:54.034513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fff61-3a6a-45c3-b127-c72a9fe3d8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:54.034783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fff61-3a6a-45c3-b127-c72a9fe3d8f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:54.098933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fff61-3a6a-45c3-b127-c72a9fe3d8f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:54.100364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fff61-3a6a-45c3-b127-c72a9fe3d8f8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:54.128866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:54.195292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 433/854] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:44:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 12:44:54.201774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33335338-b89b-4c84-8f57-8257eaf44b52 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:54.297520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33335338-b89b-4c84-8f57-8257eaf44b52 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:54.330882 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe06771-89c0-42c7-958f-a542877ad7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:54.331493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe06771-89c0-42c7-958f-a542877ad7dd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:54.331770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe06771-89c0-42c7-958f-a542877ad7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:54.332039 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe06771-89c0-42c7-958f-a542877ad7dd None admin] PATH_INFO: `/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:54.333127 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe06771-89c0-42c7-958f-a542877ad7dd None admin] RBAC: Authorizing `identity:get_project(project_id=3c9f3bcd4da84ed7945b3406a994aaa7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:54.334560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe06771-89c0-42c7-958f-a542877ad7dd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:54.336127 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 422/855] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:44:54 2022] GET /identity/v3/projects/3c9f3bcd4da84ed7945b3406a994aaa7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:54.342267 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fff61-3a6a-45c3-b127-c72a9fe3d8f8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:54.436887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fff61-3a6a-45c3-b127-c72a9fe3d8f8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:54.464686 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:54.469389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4c238-db7f-4973-8354-f0b75bf3980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:54.469981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4c238-db7f-4973-8354-f0b75bf3980c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:54.476437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4c238-db7f-4973-8354-f0b75bf3980c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:54.476437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4c238-db7f-4973-8354-f0b75bf3980c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:54.476437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4c238-db7f-4973-8354-f0b75bf3980c None admin] RBAC: Adding query filter params (role.id=ffd57ebcb27d45ef9b9d642f409cc6d7, scope.project.id=3c9f3bcd4da84ed7945b3406a994aaa7, user.id=50eadd1da25a44a583af15d561a7cc86) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:44:54.476437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4c238-db7f-4973-8354-f0b75bf3980c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:54.476437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4c238-db7f-4973-8354-f0b75bf3980c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:54.479106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 434/856] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:44:54 2022] GET /identity/v3/role_assignments?user.id=50eadd1da25a44a583af15d561a7cc86&scope.project.id=3c9f3bcd4da84ed7945b3406a994aaa7&role.id=ffd57ebcb27d45ef9b9d642f409cc6d7 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:56.599572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7f8a1-a140-475e-b06c-2af355d0e024 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:56.600049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7f8a1-a140-475e-b06c-2af355d0e024 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:56.600161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7f8a1-a140-475e-b06c-2af355d0e024 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:56.601463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 423/857] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:56.607144 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf5957-4d7e-41f1-9f26-caf3b494eb77 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:56.607562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf5957-4d7e-41f1-9f26-caf3b494eb77 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:56.607702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf5957-4d7e-41f1-9f26-caf3b494eb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:56.625429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a1cf5957-4d7e-41f1-9f26-caf3b494eb77 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:56.627590 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a1cf5957-4d7e-41f1-9f26-caf3b494eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:56.691110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 435/858] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:56.956165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7f8a1-a140-475e-b06c-2af355d0e024 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:57.002123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:57.053994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7f8a1-a140-475e-b06c-2af355d0e024 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:57.086908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd35ce1-292a-4fb1-ab2c-1a1c34ed5f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:57.087565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35ce1-292a-4fb1-ab2c-1a1c34ed5f24 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:57.087837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35ce1-292a-4fb1-ab2c-1a1c34ed5f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:57.088108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35ce1-292a-4fb1-ab2c-1a1c34ed5f24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:57.089221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd35ce1-292a-4fb1-ab2c-1a1c34ed5f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:57.090624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd35ce1-292a-4fb1-ab2c-1a1c34ed5f24 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:57.092130 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 424/859] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:57.097970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf5957-4d7e-41f1-9f26-caf3b494eb77 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:57.193897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf5957-4d7e-41f1-9f26-caf3b494eb77 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:57.218979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:57.227506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc02e09-3843-4949-8ea2-2e17e1fec780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:57.230353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc02e09-3843-4949-8ea2-2e17e1fec780 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:57.230353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc02e09-3843-4949-8ea2-2e17e1fec780 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:57.230353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc02e09-3843-4949-8ea2-2e17e1fec780 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:57.230353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc02e09-3843-4949-8ea2-2e17e1fec780 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:57.231262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc02e09-3843-4949-8ea2-2e17e1fec780 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:57.298944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 436/860] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:44:57 2022] POST /identity/v3/projects => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:44:59.463599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc630dc-13d3-4ce6-ae3d-2b7ad4c17ae3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:59.464173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc630dc-13d3-4ce6-ae3d-2b7ad4c17ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:59.464173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc630dc-13d3-4ce6-ae3d-2b7ad4c17ae3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:59.465579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 425/861] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:44:59.470807 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4764d-3e0b-48d8-9e45-8ba2f7892f65 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:59.471204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4764d-3e0b-48d8-9e45-8ba2f7892f65 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:59.471353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4764d-3e0b-48d8-9e45-8ba2f7892f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:59.488999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e1e4764d-3e0b-48d8-9e45-8ba2f7892f65 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:44:59.490219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e1e4764d-3e0b-48d8-9e45-8ba2f7892f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:44:59.554609 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 437/862] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:44:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:44:59.824904 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc630dc-13d3-4ce6-ae3d-2b7ad4c17ae3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:44:59.926294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc630dc-13d3-4ce6-ae3d-2b7ad4c17ae3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:44:59.940297 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:44:59.959546 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c31930-0c84-4054-91ad-56c9c5bcbdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:44:59.960165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31930-0c84-4054-91ad-56c9c5bcbdac None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:44:59.960445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31930-0c84-4054-91ad-56c9c5bcbdac None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:44:59.963476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31930-0c84-4054-91ad-56c9c5bcbdac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:44:59.963476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c31930-0c84-4054-91ad-56c9c5bcbdac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:44:59.963949 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c31930-0c84-4054-91ad-56c9c5bcbdac None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:44:59.965518 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 426/863] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:44:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:44:59.971203 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4764d-3e0b-48d8-9e45-8ba2f7892f65 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:00.069304 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4764d-3e0b-48d8-9e45-8ba2f7892f65 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:00.079547 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:00.102752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a08935-e377-4032-a9cd-ca4a918c63a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:00.103379 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a08935-e377-4032-a9cd-ca4a918c63a8 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:00.103654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a08935-e377-4032-a9cd-ca4a918c63a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:00.103924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a08935-e377-4032-a9cd-ca4a918c63a8 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:00.104572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a08935-e377-4032-a9cd-ca4a918c63a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:00.106018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a08935-e377-4032-a9cd-ca4a918c63a8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:00.228110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 438/864] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:44:59 2022] POST /identity/v3/users => generated 286 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:45:02.378150 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c366da3-1fb9-418c-b834-9850b6693962 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:02.378701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c366da3-1fb9-418c-b834-9850b6693962 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:02.378701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c366da3-1fb9-418c-b834-9850b6693962 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:02.380003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 427/865] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:45:02.385066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6011495-e860-4a41-950c-0f82edbfd219 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:02.385524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6011495-e860-4a41-950c-0f82edbfd219 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:02.385661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6011495-e860-4a41-950c-0f82edbfd219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:02.404005 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a6011495-e860-4a41-950c-0f82edbfd219 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:45:02.405174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a6011495-e860-4a41-950c-0f82edbfd219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:45:02.469205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 439/866] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:45:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:45:02.737867 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c366da3-1fb9-418c-b834-9850b6693962 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:02.836111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c366da3-1fb9-418c-b834-9850b6693962 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:02.869120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c66c6d-0b9d-4b0c-8b40-31ceeb0ba522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:02.869769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c66c6d-0b9d-4b0c-8b40-31ceeb0ba522 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:02.870048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c66c6d-0b9d-4b0c-8b40-31ceeb0ba522 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:02.870323 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c66c6d-0b9d-4b0c-8b40-31ceeb0ba522 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:02.880090 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c66c6d-0b9d-4b0c-8b40-31ceeb0ba522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-65c66c6d-0b9d-4b0c-8b40-31ceeb0ba522 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:45:02.884917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:45:02.886017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:45:02.886984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 428/867] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:45:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:45:02.892033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6011495-e860-4a41-950c-0f82edbfd219 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:02.981060 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:02.987536 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6011495-e860-4a41-950c-0f82edbfd219 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:03.020355 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88acb1b-962a-4f67-bfdd-7b2d111c22a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:03.020899 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88acb1b-962a-4f67-bfdd-7b2d111c22a1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:03.021219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88acb1b-962a-4f67-bfdd-7b2d111c22a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:03.021514 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88acb1b-962a-4f67-bfdd-7b2d111c22a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:03.022162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88acb1b-962a-4f67-bfdd-7b2d111c22a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:45:03.022468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88acb1b-962a-4f67-bfdd-7b2d111c22a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:03.023829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88acb1b-962a-4f67-bfdd-7b2d111c22a1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:03.031941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 440/868] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:45:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:03.037994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c66c6d-0b9d-4b0c-8b40-31ceeb0ba522 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:03.082454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:03.137436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c66c6d-0b9d-4b0c-8b40-31ceeb0ba522 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:03.170580 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbe8b1-758b-4f46-9b74-bc292b9c7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:03.171197 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbe8b1-758b-4f46-9b74-bc292b9c7aac None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:03.171479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbe8b1-758b-4f46-9b74-bc292b9c7aac None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:03.171753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbe8b1-758b-4f46-9b74-bc292b9c7aac None admin] PATH_INFO: `/v3/users/1e024be50d8a49b8bfc63c8180f0e607` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:03.184896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbe8b1-758b-4f46-9b74-bc292b9c7aac None admin] RBAC: Authorizing `identity:get_user(user_id=1e024be50d8a49b8bfc63c8180f0e607)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:03.186363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbe8b1-758b-4f46-9b74-bc292b9c7aac None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:03.187920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 429/869] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:45:03 2022] GET /identity/v3/users/1e024be50d8a49b8bfc63c8180f0e607 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:03.193709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88acb1b-962a-4f67-bfdd-7b2d111c22a1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:03.292420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88acb1b-962a-4f67-bfdd-7b2d111c22a1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:03.325751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f7001-2377-452e-97a8-ddc61aab259a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:03.326373 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f7001-2377-452e-97a8-ddc61aab259a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:03.326652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f7001-2377-452e-97a8-ddc61aab259a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:03.326925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f7001-2377-452e-97a8-ddc61aab259a None admin] PATH_INFO: `/v3/projects/1455d902121e4ea3bba5d56b27cd3f7a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:03.328009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f7001-2377-452e-97a8-ddc61aab259a None admin] RBAC: Authorizing `identity:get_project(project_id=1455d902121e4ea3bba5d56b27cd3f7a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:03.329471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f7001-2377-452e-97a8-ddc61aab259a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:03.331082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 441/870] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:45:03 2022] GET /identity/v3/projects/1455d902121e4ea3bba5d56b27cd3f7a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:03.337063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbe8b1-758b-4f46-9b74-bc292b9c7aac None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:03.432483 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:03.435314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbe8b1-758b-4f46-9b74-bc292b9c7aac None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:03.467606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3387ca-dfdf-4063-9ccd-5c1cb7b92f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:03.468244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3387ca-dfdf-4063-9ccd-5c1cb7b92f72 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:03.468520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3387ca-dfdf-4063-9ccd-5c1cb7b92f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:03.468799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3387ca-dfdf-4063-9ccd-5c1cb7b92f72 None admin] PATH_INFO: `/v3/projects/1455d902121e4ea3bba5d56b27cd3f7a/users/1e024be50d8a49b8bfc63c8180f0e607/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:03.470690 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3387ca-dfdf-4063-9ccd-5c1cb7b92f72 None admin] RBAC: Authorizing `identity:create_grant(project_id=1455d902121e4ea3bba5d56b27cd3f7a, user_id=1e024be50d8a49b8bfc63c8180f0e607, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:03.472104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3387ca-dfdf-4063-9ccd-5c1cb7b92f72 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:03.576843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 430/871] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:45:03 2022] PUT /identity/v3/projects/1455d902121e4ea3bba5d56b27cd3f7a/users/1e024be50d8a49b8bfc63c8180f0e607/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:45:05.684064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfba58c-606e-4ae4-869e-b65f8b248cf7 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:05.684607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfba58c-606e-4ae4-869e-b65f8b248cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:05.684607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfba58c-606e-4ae4-869e-b65f8b248cf7 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:05.685978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 442/872] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:45:05.691118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670602b-67ba-4497-ac69-f037a209e932 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:05.691481 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670602b-67ba-4497-ac69-f037a209e932 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:05.691677 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670602b-67ba-4497-ac69-f037a209e932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:05.710121 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9670602b-67ba-4497-ac69-f037a209e932 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:45:05.711303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9670602b-67ba-4497-ac69-f037a209e932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:45:05.775223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 431/873] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:45:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:45:06.043099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfba58c-606e-4ae4-869e-b65f8b248cf7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:06.069237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:06.143179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfba58c-606e-4ae4-869e-b65f8b248cf7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:06.176345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00beabb5-5dd5-4de0-8675-026d3c928bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:06.176969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00beabb5-5dd5-4de0-8675-026d3c928bbb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:06.177270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00beabb5-5dd5-4de0-8675-026d3c928bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:06.177542 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00beabb5-5dd5-4de0-8675-026d3c928bbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:06.187251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00beabb5-5dd5-4de0-8675-026d3c928bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-00beabb5-5dd5-4de0-8675-026d3c928bbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:45:06.191977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:45:06.193119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:45:06.194272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 443/874] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:45:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:45:06.199306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670602b-67ba-4497-ac69-f037a209e932 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:06.284594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:06.293875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670602b-67ba-4497-ac69-f037a209e932 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:06.327827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e9c57-778b-4db6-8509-865d56a22336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:06.328435 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e9c57-778b-4db6-8509-865d56a22336 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:06.328717 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e9c57-778b-4db6-8509-865d56a22336 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:06.328994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e9c57-778b-4db6-8509-865d56a22336 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:06.329706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e9c57-778b-4db6-8509-865d56a22336 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:45:06.330014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e9c57-778b-4db6-8509-865d56a22336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:06.331382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e9c57-778b-4db6-8509-865d56a22336 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:06.339774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 432/875] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:45:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:06.345987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00beabb5-5dd5-4de0-8675-026d3c928bbb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:06.427701 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:06.443517 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00beabb5-5dd5-4de0-8675-026d3c928bbb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:06.477048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb73c11-5967-40e7-aeed-b585a8f5a211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:06.478306 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb73c11-5967-40e7-aeed-b585a8f5a211 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:06.478581 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb73c11-5967-40e7-aeed-b585a8f5a211 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:06.478851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb73c11-5967-40e7-aeed-b585a8f5a211 None admin] PATH_INFO: `/v3/users/1e024be50d8a49b8bfc63c8180f0e607` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:06.479948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb73c11-5967-40e7-aeed-b585a8f5a211 None admin] RBAC: Authorizing `identity:get_user(user_id=1e024be50d8a49b8bfc63c8180f0e607)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:06.481405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb73c11-5967-40e7-aeed-b585a8f5a211 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:06.482947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 444/876] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:45:06 2022] GET /identity/v3/users/1e024be50d8a49b8bfc63c8180f0e607 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:06.488959 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e9c57-778b-4db6-8509-865d56a22336 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:06.587449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e9c57-778b-4db6-8509-865d56a22336 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:06.621446 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff85b7-b1d9-4b94-91dd-df7aa5eee315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:06.622141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff85b7-b1d9-4b94-91dd-df7aa5eee315 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:06.622431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff85b7-b1d9-4b94-91dd-df7aa5eee315 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:06.622708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff85b7-b1d9-4b94-91dd-df7aa5eee315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:06.643178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:06.689088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff85b7-b1d9-4b94-91dd-df7aa5eee315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:06.690573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff85b7-b1d9-4b94-91dd-df7aa5eee315 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:06.786280 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 433/877] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:45:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 12:45:06.792930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb73c11-5967-40e7-aeed-b585a8f5a211 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:06.887907 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb73c11-5967-40e7-aeed-b585a8f5a211 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:06.921039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7018fe-4ce3-4836-a54c-7baafd045aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:06.921680 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7018fe-4ce3-4836-a54c-7baafd045aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:06.921956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7018fe-4ce3-4836-a54c-7baafd045aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:06.922227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7018fe-4ce3-4836-a54c-7baafd045aa7 None admin] PATH_INFO: `/v3/projects/1455d902121e4ea3bba5d56b27cd3f7a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:06.923308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7018fe-4ce3-4836-a54c-7baafd045aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=1455d902121e4ea3bba5d56b27cd3f7a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:06.924728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7018fe-4ce3-4836-a54c-7baafd045aa7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:06.926351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 445/878] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:45:06 2022] GET /identity/v3/projects/1455d902121e4ea3bba5d56b27cd3f7a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:06.932492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff85b7-b1d9-4b94-91dd-df7aa5eee315 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:06.982839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:07.027672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff85b7-b1d9-4b94-91dd-df7aa5eee315 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:07.060748 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1e07f-5779-43c9-ae3b-c3aaa8133c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:07.061370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1e07f-5779-43c9-ae3b-c3aaa8133c60 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:07.061647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1e07f-5779-43c9-ae3b-c3aaa8133c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:07.061915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1e07f-5779-43c9-ae3b-c3aaa8133c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:07.062621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1e07f-5779-43c9-ae3b-c3aaa8133c60 None admin] RBAC: Adding query filter params (role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.project.id=1455d902121e4ea3bba5d56b27cd3f7a, user.id=1e024be50d8a49b8bfc63c8180f0e607) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:45:07.062925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1e07f-5779-43c9-ae3b-c3aaa8133c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:07.064331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1e07f-5779-43c9-ae3b-c3aaa8133c60 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:07.070176 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 434/879] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:45:06 2022] GET /identity/v3/role_assignments?user.id=1e024be50d8a49b8bfc63c8180f0e607&scope.project.id=1455d902121e4ea3bba5d56b27cd3f7a&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:09.212813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7a852-e35c-4668-a95b-b6680528051f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:09.213384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7a852-e35c-4668-a95b-b6680528051f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:09.213384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7a852-e35c-4668-a95b-b6680528051f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:09.214704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 446/880] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:45:09.219863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a585de6-994f-41b8-bbcd-badf1abb5a74 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:09.220276 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a585de6-994f-41b8-bbcd-badf1abb5a74 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:09.220417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a585de6-994f-41b8-bbcd-badf1abb5a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:09.238911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7a585de6-994f-41b8-bbcd-badf1abb5a74 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:45:09.240101 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7a585de6-994f-41b8-bbcd-badf1abb5a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:45:09.308311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 435/881] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:45:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:45:09.580973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7a852-e35c-4668-a95b-b6680528051f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:09.602066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:09.679538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7a852-e35c-4668-a95b-b6680528051f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:09.712441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd80f4a-8e98-4b3d-a92e-1e1a20111591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:09.713030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd80f4a-8e98-4b3d-a92e-1e1a20111591 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:09.713345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd80f4a-8e98-4b3d-a92e-1e1a20111591 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:09.713622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd80f4a-8e98-4b3d-a92e-1e1a20111591 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:09.714293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd80f4a-8e98-4b3d-a92e-1e1a20111591 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:09.715658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd80f4a-8e98-4b3d-a92e-1e1a20111591 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:09.766240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 447/882] 10.222.0.30 () {60 vars in 1138 bytes} [Sat Oct 15 12:45:09 2022] POST /identity/v3/domains => generated 267 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:45:11.918187 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d97a0-df99-4ef3-8450-20a1aa13c491 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:11.918700 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d97a0-df99-4ef3-8450-20a1aa13c491 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:11.918700 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d97a0-df99-4ef3-8450-20a1aa13c491 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:11.919983 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 436/883] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:45:11.925444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c238-7b6a-47e6-99df-4ef9c25b76c4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:11.925818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c238-7b6a-47e6-99df-4ef9c25b76c4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:11.925997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c238-7b6a-47e6-99df-4ef9c25b76c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:11.944121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0147c238-7b6a-47e6-99df-4ef9c25b76c4 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:45:11.945349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0147c238-7b6a-47e6-99df-4ef9c25b76c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:45:12.010106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 448/884] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:45:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:45:12.279303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d97a0-df99-4ef3-8450-20a1aa13c491 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:12.326009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:12.377771 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d97a0-df99-4ef3-8450-20a1aa13c491 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:12.410338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a3fd3-c87c-4cc4-95f4-ebd14252aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:12.411214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a3fd3-c87c-4cc4-95f4-ebd14252aefe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:12.411491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a3fd3-c87c-4cc4-95f4-ebd14252aefe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:12.411761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a3fd3-c87c-4cc4-95f4-ebd14252aefe None admin] PATH_INFO: `/v3/domains/f58baf77d40c49a0ae58196437a7461a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:12.422279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a3fd3-c87c-4cc4-95f4-ebd14252aefe None admin] RBAC: Authorizing `identity:get_domain(domain_id=f58baf77d40c49a0ae58196437a7461a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:12.423710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a3fd3-c87c-4cc4-95f4-ebd14252aefe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:12.425280 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 437/885] 10.222.0.30 () {56 vars in 1183 bytes} [Sat Oct 15 12:45:12 2022] GET /identity/v3/domains/f58baf77d40c49a0ae58196437a7461a => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:12.431242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147c238-7b6a-47e6-99df-4ef9c25b76c4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:12.442427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:12.527082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147c238-7b6a-47e6-99df-4ef9c25b76c4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:12.560837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984c163-cac1-4635-83f7-b5f2bdd42c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:12.561480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984c163-cac1-4635-83f7-b5f2bdd42c6e None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:12.561758 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984c163-cac1-4635-83f7-b5f2bdd42c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:12.562028 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984c163-cac1-4635-83f7-b5f2bdd42c6e None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:12.562675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984c163-cac1-4635-83f7-b5f2bdd42c6e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:12.564113 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984c163-cac1-4635-83f7-b5f2bdd42c6e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:12.641979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 449/886] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:45:12 2022] POST /identity/v3/projects => generated 361 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:45:14.810612 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f5a2a-0f80-42ab-a81b-cd128391a0f3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:14.811110 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f5a2a-0f80-42ab-a81b-cd128391a0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:14.811110 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f5a2a-0f80-42ab-a81b-cd128391a0f3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:14.812411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 438/887] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:45:14.817656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448f7b4-6799-4b79-9778-71f105bb3ea0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:14.818086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448f7b4-6799-4b79-9778-71f105bb3ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:14.818237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448f7b4-6799-4b79-9778-71f105bb3ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:14.836762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9448f7b4-6799-4b79-9778-71f105bb3ea0 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:45:14.838616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9448f7b4-6799-4b79-9778-71f105bb3ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:45:14.893055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:14.903631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 450/888] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:45:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:45:15.176075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f5a2a-0f80-42ab-a81b-cd128391a0f3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:15.273945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f5a2a-0f80-42ab-a81b-cd128391a0f3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:15.281094 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:15.307828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee4de6-095d-4ec3-bcd9-f6d08306e5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:15.308396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee4de6-095d-4ec3-bcd9-f6d08306e5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:15.308674 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee4de6-095d-4ec3-bcd9-f6d08306e5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:15.308949 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee4de6-095d-4ec3-bcd9-f6d08306e5b3 None admin] PATH_INFO: `/v3/domains/f58baf77d40c49a0ae58196437a7461a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:15.310164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee4de6-095d-4ec3-bcd9-f6d08306e5b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f58baf77d40c49a0ae58196437a7461a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:15.311583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee4de6-095d-4ec3-bcd9-f6d08306e5b3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:15.313175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 439/889] 10.222.0.30 () {56 vars in 1183 bytes} [Sat Oct 15 12:45:15 2022] GET /identity/v3/domains/f58baf77d40c49a0ae58196437a7461a => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:15.319143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448f7b4-6799-4b79-9778-71f105bb3ea0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:15.416666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448f7b4-6799-4b79-9778-71f105bb3ea0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:15.451082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355c59f-7b66-44e2-bc8b-80e83624afc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:15.451631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355c59f-7b66-44e2-bc8b-80e83624afc6 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:15.451905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355c59f-7b66-44e2-bc8b-80e83624afc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:15.452174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355c59f-7b66-44e2-bc8b-80e83624afc6 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:15.452812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355c59f-7b66-44e2-bc8b-80e83624afc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:15.454277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355c59f-7b66-44e2-bc8b-80e83624afc6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:15.514029 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 451/890] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:45:15 2022] POST /identity/v3/users => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:45:17.744092 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12c24b-17a0-43d4-b1ba-719fde3bc157 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:17.744654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12c24b-17a0-43d4-b1ba-719fde3bc157 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:17.744654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12c24b-17a0-43d4-b1ba-719fde3bc157 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:17.746018 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 440/891] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:45:17.751124 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3e89e-52ef-417c-b4ee-9a9a51409749 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:17.751556 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3e89e-52ef-417c-b4ee-9a9a51409749 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:17.751666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3e89e-52ef-417c-b4ee-9a9a51409749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:17.769976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-94a3e89e-52ef-417c-b4ee-9a9a51409749 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:45:17.771148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-94a3e89e-52ef-417c-b4ee-9a9a51409749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:45:17.813714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:17.836611 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 452/892] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:45:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:45:18.113788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12c24b-17a0-43d4-b1ba-719fde3bc157 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:18.222251 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12c24b-17a0-43d4-b1ba-719fde3bc157 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:18.255386 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cabe1-f568-47a4-93bd-cea355ad427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:18.256006 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cabe1-f568-47a4-93bd-cea355ad427a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:18.256283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cabe1-f568-47a4-93bd-cea355ad427a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:18.256557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cabe1-f568-47a4-93bd-cea355ad427a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:18.267154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cabe1-f568-47a4-93bd-cea355ad427a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-652cabe1-f568-47a4-93bd-cea355ad427a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:45:18.271940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:45:18.273023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:45:18.273980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 441/893] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:45:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:45:18.279181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3e89e-52ef-417c-b4ee-9a9a51409749 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:18.374729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3e89e-52ef-417c-b4ee-9a9a51409749 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:18.407516 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc89761-4234-4205-ba8b-8a044c160119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:18.408117 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc89761-4234-4205-ba8b-8a044c160119 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:18.408393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc89761-4234-4205-ba8b-8a044c160119 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:18.408663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc89761-4234-4205-ba8b-8a044c160119 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:18.409350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc89761-4234-4205-ba8b-8a044c160119 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:45:18.409654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc89761-4234-4205-ba8b-8a044c160119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:18.411259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc89761-4234-4205-ba8b-8a044c160119 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:18.418946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 453/894] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:45:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:18.424709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cabe1-f568-47a4-93bd-cea355ad427a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:18.459352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:18.522359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cabe1-f568-47a4-93bd-cea355ad427a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:18.555120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0659f-d716-4202-b5f0-f84a42fc8dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:18.556347 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0659f-d716-4202-b5f0-f84a42fc8dfa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:18.556616 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0659f-d716-4202-b5f0-f84a42fc8dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:18.556885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0659f-d716-4202-b5f0-f84a42fc8dfa None admin] PATH_INFO: `/v3/users/03ecabc3e430422097ba524bb7d999ff` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:18.569716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0659f-d716-4202-b5f0-f84a42fc8dfa None admin] RBAC: Authorizing `identity:get_user(user_id=03ecabc3e430422097ba524bb7d999ff)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:18.571196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0659f-d716-4202-b5f0-f84a42fc8dfa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:18.572772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 442/895] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:45:18 2022] GET /identity/v3/users/03ecabc3e430422097ba524bb7d999ff => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:18.578510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc89761-4234-4205-ba8b-8a044c160119 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:18.619374 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:18.675258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc89761-4234-4205-ba8b-8a044c160119 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:18.707137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89bb17-1388-4bd1-995b-a46ae93b6ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:18.707751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89bb17-1388-4bd1-995b-a46ae93b6ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:18.708030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89bb17-1388-4bd1-995b-a46ae93b6ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:18.708299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89bb17-1388-4bd1-995b-a46ae93b6ca7 None admin] PATH_INFO: `/v3/projects/d93ae5b925564cd9b6d10891db829abd` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:18.709423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89bb17-1388-4bd1-995b-a46ae93b6ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=d93ae5b925564cd9b6d10891db829abd)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:18.710838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89bb17-1388-4bd1-995b-a46ae93b6ca7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:18.712468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 454/896] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:45:18 2022] GET /identity/v3/projects/d93ae5b925564cd9b6d10891db829abd => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:18.718504 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0659f-d716-4202-b5f0-f84a42fc8dfa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:18.806788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:18.815069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0659f-d716-4202-b5f0-f84a42fc8dfa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:18.847812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38578d1b-0a7d-405a-b941-acec25e19c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:18.848453 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38578d1b-0a7d-405a-b941-acec25e19c50 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:18.848729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38578d1b-0a7d-405a-b941-acec25e19c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:18.849002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38578d1b-0a7d-405a-b941-acec25e19c50 None admin] PATH_INFO: `/v3/projects/d93ae5b925564cd9b6d10891db829abd/users/03ecabc3e430422097ba524bb7d999ff/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:18.850909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38578d1b-0a7d-405a-b941-acec25e19c50 None admin] RBAC: Authorizing `identity:create_grant(project_id=d93ae5b925564cd9b6d10891db829abd, user_id=03ecabc3e430422097ba524bb7d999ff, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:18.852342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38578d1b-0a7d-405a-b941-acec25e19c50 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:18.907926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 443/897] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:45:18 2022] PUT /identity/v3/projects/d93ae5b925564cd9b6d10891db829abd/users/03ecabc3e430422097ba524bb7d999ff/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:45:20.996419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722690e-5bed-4cf9-8bf5-853b062e4561 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:20.996918 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722690e-5bed-4cf9-8bf5-853b062e4561 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:20.996918 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722690e-5bed-4cf9-8bf5-853b062e4561 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:20.998253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 455/898] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:45:21.003536 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bec256-3695-44da-8cda-6937ff7fb1f0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:21.003972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bec256-3695-44da-8cda-6937ff7fb1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:21.004064 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bec256-3695-44da-8cda-6937ff7fb1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:21.022547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-24bec256-3695-44da-8cda-6937ff7fb1f0 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:45:21.023724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-24bec256-3695-44da-8cda-6937ff7fb1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:45:21.088193 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 444/899] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:45:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:45:21.356817 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722690e-5bed-4cf9-8bf5-853b062e4561 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:21.455976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722690e-5bed-4cf9-8bf5-853b062e4561 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:21.465436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:21.489426 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab62f7-0d5c-444a-a828-5334a99947d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:21.490075 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab62f7-0d5c-444a-a828-5334a99947d2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:21.490355 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab62f7-0d5c-444a-a828-5334a99947d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:21.490629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab62f7-0d5c-444a-a828-5334a99947d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:21.501330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab62f7-0d5c-444a-a828-5334a99947d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-83ab62f7-0d5c-444a-a828-5334a99947d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:45:21.506034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:45:21.507154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:45:21.508017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 456/900] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:45:21.514022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bec256-3695-44da-8cda-6937ff7fb1f0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:21.592382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:21.609402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bec256-3695-44da-8cda-6937ff7fb1f0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:21.642004 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486c1f0-36a4-4f62-9a5c-674c2a742302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:21.642602 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486c1f0-36a4-4f62-9a5c-674c2a742302 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:21.642878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486c1f0-36a4-4f62-9a5c-674c2a742302 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:21.643254 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486c1f0-36a4-4f62-9a5c-674c2a742302 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:21.643908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486c1f0-36a4-4f62-9a5c-674c2a742302 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:45:21.644209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486c1f0-36a4-4f62-9a5c-674c2a742302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:21.645590 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486c1f0-36a4-4f62-9a5c-674c2a742302 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:21.654232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 445/901] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:45:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:21.660226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab62f7-0d5c-444a-a828-5334a99947d2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:21.756244 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab62f7-0d5c-444a-a828-5334a99947d2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:21.790375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f28989-7280-4438-b068-0d319fd32c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:21.790981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f28989-7280-4438-b068-0d319fd32c48 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:21.791260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f28989-7280-4438-b068-0d319fd32c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:21.791533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f28989-7280-4438-b068-0d319fd32c48 None admin] PATH_INFO: `/v3/users/03ecabc3e430422097ba524bb7d999ff` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:21.792630 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f28989-7280-4438-b068-0d319fd32c48 None admin] RBAC: Authorizing `identity:get_user(user_id=03ecabc3e430422097ba524bb7d999ff)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:21.794088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f28989-7280-4438-b068-0d319fd32c48 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:21.795622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 457/902] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 12:45:21 2022] GET /identity/v3/users/03ecabc3e430422097ba524bb7d999ff => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:21.801354 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486c1f0-36a4-4f62-9a5c-674c2a742302 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:21.899260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486c1f0-36a4-4f62-9a5c-674c2a742302 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:21.932011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea4bf4-21c3-406e-8b87-58ffa994d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:21.932608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea4bf4-21c3-406e-8b87-58ffa994d99e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:21.932889 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea4bf4-21c3-406e-8b87-58ffa994d99e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:21.933169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea4bf4-21c3-406e-8b87-58ffa994d99e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:21.947400 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:21.998667 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea4bf4-21c3-406e-8b87-58ffa994d99e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:22.000101 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea4bf4-21c3-406e-8b87-58ffa994d99e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:22.094942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 446/903] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:45:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 12:45:22.100942 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f28989-7280-4438-b068-0d319fd32c48 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:22.135039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:22.198077 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f28989-7280-4438-b068-0d319fd32c48 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:22.230306 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682ac05-0206-4a2f-90ed-cb2bed217f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:22.230927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682ac05-0206-4a2f-90ed-cb2bed217f46 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:22.231205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682ac05-0206-4a2f-90ed-cb2bed217f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:22.231477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682ac05-0206-4a2f-90ed-cb2bed217f46 None admin] PATH_INFO: `/v3/projects/d93ae5b925564cd9b6d10891db829abd` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:22.232562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682ac05-0206-4a2f-90ed-cb2bed217f46 None admin] RBAC: Authorizing `identity:get_project(project_id=d93ae5b925564cd9b6d10891db829abd)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:22.234013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682ac05-0206-4a2f-90ed-cb2bed217f46 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:22.235594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 458/904] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 12:45:22 2022] GET /identity/v3/projects/d93ae5b925564cd9b6d10891db829abd => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:22.241797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea4bf4-21c3-406e-8b87-58ffa994d99e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:45:22.285435 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:22.336265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea4bf4-21c3-406e-8b87-58ffa994d99e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:45:22.368796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746a6d0-031f-44c8-8e21-b03951075c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:45:22.369416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746a6d0-031f-44c8-8e21-b03951075c82 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:45:22.369694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746a6d0-031f-44c8-8e21-b03951075c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:45:22.369964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746a6d0-031f-44c8-8e21-b03951075c82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:45:22.370671 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746a6d0-031f-44c8-8e21-b03951075c82 None admin] RBAC: Adding query filter params (role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.project.id=d93ae5b925564cd9b6d10891db829abd, user.id=03ecabc3e430422097ba524bb7d999ff) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:45:22.370974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746a6d0-031f-44c8-8e21-b03951075c82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:45:22.372376 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746a6d0-031f-44c8-8e21-b03951075c82 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:45:22.377835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 447/905] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:45:22 2022] GET /identity/v3/role_assignments?user.id=03ecabc3e430422097ba524bb7d999ff&scope.project.id=d93ae5b925564cd9b6d10891db829abd&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:45:32.225733 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=48 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:45:32.375405 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=46 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:41.636357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad9c7f-f1a6-42e3-a822-bccee5822cfd None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:41.636938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad9c7f-f1a6-42e3-a822-bccee5822cfd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:41.636938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad9c7f-f1a6-42e3-a822-bccee5822cfd None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:41.638282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 459/906] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:46:41.643731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba26632-a52a-45db-8465-0d5f44364dd1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:41.644164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba26632-a52a-45db-8465-0d5f44364dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:41.644863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba26632-a52a-45db-8465-0d5f44364dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:41.662703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-aba26632-a52a-45db-8465-0d5f44364dd1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:46:41.664151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aba26632-a52a-45db-8465-0d5f44364dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:46:41.728642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 448/907] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:46:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:46:42.003124 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad9c7f-f1a6-42e3-a822-bccee5822cfd None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:42.102881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad9c7f-f1a6-42e3-a822-bccee5822cfd None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:42.134839 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a6cfc-88b8-4edd-a230-bce5d9c7e066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:42.135497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a6cfc-88b8-4edd-a230-bce5d9c7e066 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:42.135780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a6cfc-88b8-4edd-a230-bce5d9c7e066 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:42.136054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a6cfc-88b8-4edd-a230-bce5d9c7e066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:42.146481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a6cfc-88b8-4edd-a230-bce5d9c7e066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:42.147998 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a6cfc-88b8-4edd-a230-bce5d9c7e066 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e81a6cfc-88b8-4edd-a230-bce5d9c7e066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:46:42.162842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:42.164260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:46:42.165656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:46:42.165656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:42.165656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:42.165656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 460/908] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:42.170494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba26632-a52a-45db-8465-0d5f44364dd1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:42.266508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba26632-a52a-45db-8465-0d5f44364dd1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:42.295051 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:42.300857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c7e03-e8fc-4709-95ef-da6430b87bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:42.301566 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c7e03-e8fc-4709-95ef-da6430b87bbc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:42.301847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c7e03-e8fc-4709-95ef-da6430b87bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:42.302119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c7e03-e8fc-4709-95ef-da6430b87bbc None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:42.302753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c7e03-e8fc-4709-95ef-da6430b87bbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:42.303056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c7e03-e8fc-4709-95ef-da6430b87bbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:42.304409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c7e03-e8fc-4709-95ef-da6430b87bbc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:42.315057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 449/909] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:42.321502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a6cfc-88b8-4edd-a230-bce5d9c7e066 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:42.372729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:42.419419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a6cfc-88b8-4edd-a230-bce5d9c7e066 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:42.451850 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f3876-ae83-4e8a-bc32-22d32dbb5bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:42.452448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f3876-ae83-4e8a-bc32-22d32dbb5bad None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:42.452722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f3876-ae83-4e8a-bc32-22d32dbb5bad None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:42.452988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f3876-ae83-4e8a-bc32-22d32dbb5bad None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:42.453661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f3876-ae83-4e8a-bc32-22d32dbb5bad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:42.455083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f3876-ae83-4e8a-bc32-22d32dbb5bad None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:42.577819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 461/910] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 12:46:42 2022] POST /identity/v3/users => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:46:44.728331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75a1fe-1301-4e5c-8714-6b2b6bdef1c0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:44.728871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75a1fe-1301-4e5c-8714-6b2b6bdef1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:44.728871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75a1fe-1301-4e5c-8714-6b2b6bdef1c0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:44.730325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 450/911] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:46:44.735613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecc65e-7d9f-4116-877d-233c0e60d941 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:44.735989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecc65e-7d9f-4116-877d-233c0e60d941 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:44.736171 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecc65e-7d9f-4116-877d-233c0e60d941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:44.754658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-92ecc65e-7d9f-4116-877d-233c0e60d941 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:46:44.755843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-92ecc65e-7d9f-4116-877d-233c0e60d941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:46:44.821752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 462/912] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:46:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:46:45.089901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75a1fe-1301-4e5c-8714-6b2b6bdef1c0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:45.191835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75a1fe-1301-4e5c-8714-6b2b6bdef1c0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:45.224903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea187-8f03-466d-8906-bf15eb52392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:45.225564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea187-8f03-466d-8906-bf15eb52392c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:45.225844 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea187-8f03-466d-8906-bf15eb52392c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:45.226117 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea187-8f03-466d-8906-bf15eb52392c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:45.237081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dea187-8f03-466d-8906-bf15eb52392c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e8dea187-8f03-466d-8906-bf15eb52392c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:46:45.242461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:45.248774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:45.249695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 451/913] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:46:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:45.250441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecc65e-7d9f-4116-877d-233c0e60d941 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:45.288571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:45.344906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecc65e-7d9f-4116-877d-233c0e60d941 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:45.378660 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90498-0ff5-47ee-a0ce-e48a74b46364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:45.379461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90498-0ff5-47ee-a0ce-e48a74b46364 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:45.379723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90498-0ff5-47ee-a0ce-e48a74b46364 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:45.379993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90498-0ff5-47ee-a0ce-e48a74b46364 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:45.380651 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90498-0ff5-47ee-a0ce-e48a74b46364 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:45.380955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90498-0ff5-47ee-a0ce-e48a74b46364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:45.382371 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90498-0ff5-47ee-a0ce-e48a74b46364 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:45.390019 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 463/914] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:46:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:45.395886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea187-8f03-466d-8906-bf15eb52392c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:45.446068 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:45.496158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea187-8f03-466d-8906-bf15eb52392c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:45.529007 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bac06-db5a-4f42-93f3-f614a23698a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:45.529668 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bac06-db5a-4f42-93f3-f614a23698a3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:45.529951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bac06-db5a-4f42-93f3-f614a23698a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:45.530230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bac06-db5a-4f42-93f3-f614a23698a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:45.541336 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bac06-db5a-4f42-93f3-f614a23698a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:45.545838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bac06-db5a-4f42-93f3-f614a23698a3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-789bac06-db5a-4f42-93f3-f614a23698a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:46:45.556491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:45.557664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:46:45.559458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:46:45.559458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:45.559458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:45.559458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 452/915] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:45.563381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90498-0ff5-47ee-a0ce-e48a74b46364 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:45.659181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:45.662795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90498-0ff5-47ee-a0ce-e48a74b46364 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:45.696983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71c29f-efb4-4ecc-80f9-2eb0c96e756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:45.697625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71c29f-efb4-4ecc-80f9-2eb0c96e756a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:45.697904 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71c29f-efb4-4ecc-80f9-2eb0c96e756a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:45.698177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71c29f-efb4-4ecc-80f9-2eb0c96e756a None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:45.698841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71c29f-efb4-4ecc-80f9-2eb0c96e756a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:45.699145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71c29f-efb4-4ecc-80f9-2eb0c96e756a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:45.700506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71c29f-efb4-4ecc-80f9-2eb0c96e756a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:45.713139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 464/916] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:45.718834 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bac06-db5a-4f42-93f3-f614a23698a3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:45.819847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bac06-db5a-4f42-93f3-f614a23698a3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:45.821517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:45.852358 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe58e9-cac8-4723-9c59-180cb6078e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:45.852980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe58e9-cac8-4723-9c59-180cb6078e3a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:45.853285 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe58e9-cac8-4723-9c59-180cb6078e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:45.853561 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe58e9-cac8-4723-9c59-180cb6078e3a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:45.860685 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe58e9-cac8-4723-9c59-180cb6078e3a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:45.862154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe58e9-cac8-4723-9c59-180cb6078e3a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-92fe58e9-cac8-4723-9c59-180cb6078e3a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:46:45.873618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:45.874627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:45.875699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 453/917] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:45.881457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71c29f-efb4-4ecc-80f9-2eb0c96e756a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:45.980001 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71c29f-efb4-4ecc-80f9-2eb0c96e756a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:46.012502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de26fa-865c-4e6c-a0ee-a3009f619736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:46.013091 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de26fa-865c-4e6c-a0ee-a3009f619736 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:46.013401 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de26fa-865c-4e6c-a0ee-a3009f619736 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:46.013677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de26fa-865c-4e6c-a0ee-a3009f619736 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:46.014392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de26fa-865c-4e6c-a0ee-a3009f619736 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:46.014698 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de26fa-865c-4e6c-a0ee-a3009f619736 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:46.016187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de26fa-865c-4e6c-a0ee-a3009f619736 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:46.032788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 465/918] 10.222.0.30 () {56 vars in 1143 bytes} [Sat Oct 15 12:46:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:46.039716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe58e9-cac8-4723-9c59-180cb6078e3a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:46.136321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe58e9-cac8-4723-9c59-180cb6078e3a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:46.169606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef432a2-9ca2-408e-9e47-6c40fe864130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:46.170272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef432a2-9ca2-408e-9e47-6c40fe864130 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:46.170547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef432a2-9ca2-408e-9e47-6c40fe864130 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:46.170820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef432a2-9ca2-408e-9e47-6c40fe864130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:46.181563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef432a2-9ca2-408e-9e47-6c40fe864130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:46.183075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef432a2-9ca2-408e-9e47-6c40fe864130 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9ef432a2-9ca2-408e-9e47-6c40fe864130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:46:46.196794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:46.197947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:46:46.199030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:46:46.199030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:46.199030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:46.199030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 454/919] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:46.203626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de26fa-865c-4e6c-a0ee-a3009f619736 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:46.224751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:46.304283 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de26fa-865c-4e6c-a0ee-a3009f619736 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:46.337068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71155ecd-c99c-49dc-8a1e-eebd54fbd0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:46.337700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71155ecd-c99c-49dc-8a1e-eebd54fbd0fe None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:46.337982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71155ecd-c99c-49dc-8a1e-eebd54fbd0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:46.338257 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71155ecd-c99c-49dc-8a1e-eebd54fbd0fe None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:46.338888 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71155ecd-c99c-49dc-8a1e-eebd54fbd0fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:46.339193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71155ecd-c99c-49dc-8a1e-eebd54fbd0fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:46.340557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71155ecd-c99c-49dc-8a1e-eebd54fbd0fe None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:46.352184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 466/920] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:46.358120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef432a2-9ca2-408e-9e47-6c40fe864130 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:46.374227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:46.457580 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef432a2-9ca2-408e-9e47-6c40fe864130 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:46.490760 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66dd49-361c-4265-a5c0-ae316dcfc849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:46.491389 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66dd49-361c-4265-a5c0-ae316dcfc849 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:46.491669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66dd49-361c-4265-a5c0-ae316dcfc849 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:46.491942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66dd49-361c-4265-a5c0-ae316dcfc849 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:46.498486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66dd49-361c-4265-a5c0-ae316dcfc849 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:46.499932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66dd49-361c-4265-a5c0-ae316dcfc849 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6f66dd49-361c-4265-a5c0-ae316dcfc849 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:46:46.511905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:46.518470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:46.520063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:46:46.520063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:46:46.520063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:46:46.520063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:46:46.520063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:46:46.520063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:46:46.520063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:46:46.520063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:46.520063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 455/921] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:46.520770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71155ecd-c99c-49dc-8a1e-eebd54fbd0fe None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:46.599171 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:46.619369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71155ecd-c99c-49dc-8a1e-eebd54fbd0fe None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:46.651850 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b44159-3c0f-4845-94d5-6a156adc5808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:46.652451 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b44159-3c0f-4845-94d5-6a156adc5808 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:46.652745 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b44159-3c0f-4845-94d5-6a156adc5808 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:46.653000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b44159-3c0f-4845-94d5-6a156adc5808 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:46.653706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b44159-3c0f-4845-94d5-6a156adc5808 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:46.654008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b44159-3c0f-4845-94d5-6a156adc5808 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:46.655419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b44159-3c0f-4845-94d5-6a156adc5808 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:46.667846 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 467/922] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:46.674096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66dd49-361c-4265-a5c0-ae316dcfc849 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:46.747432 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:46.773015 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66dd49-361c-4265-a5c0-ae316dcfc849 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:46.805052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47aaf70-2c86-4f99-bfdc-69453432f05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:46.805731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aaf70-2c86-4f99-bfdc-69453432f05e None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:46.806011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aaf70-2c86-4f99-bfdc-69453432f05e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:46.806283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aaf70-2c86-4f99-bfdc-69453432f05e None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/18df1f401bb2494299eda9582c85a04f/roles/5a5e7aced6d84267bd9e1fa0c1016420` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:46.820754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47aaf70-2c86-4f99-bfdc-69453432f05e None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=18df1f401bb2494299eda9582c85a04f, role_id=5a5e7aced6d84267bd9e1fa0c1016420)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:46.822178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47aaf70-2c86-4f99-bfdc-69453432f05e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:46.914389 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 456/923] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:46:46 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/18df1f401bb2494299eda9582c85a04f/roles/5a5e7aced6d84267bd9e1fa0c1016420 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:46:49.048016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39807c-933e-4df1-a2c1-2e05b98835c1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:49.049326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39807c-933e-4df1-a2c1-2e05b98835c1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:49.049326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39807c-933e-4df1-a2c1-2e05b98835c1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:49.049952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 468/924] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:46:49.055733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af6f91-44c4-4d60-ad13-262b45a91c45 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:49.056106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af6f91-44c4-4d60-ad13-262b45a91c45 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:49.056416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af6f91-44c4-4d60-ad13-262b45a91c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:49.074275 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b2af6f91-44c4-4d60-ad13-262b45a91c45 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:46:49.078124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b2af6f91-44c4-4d60-ad13-262b45a91c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:46:49.143605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 457/925] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:46:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:46:49.416932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39807c-933e-4df1-a2c1-2e05b98835c1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:49.520154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39807c-933e-4df1-a2c1-2e05b98835c1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:49.555613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea6d65-cc02-45c4-8ac2-bf4e042b62c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:49.556238 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea6d65-cc02-45c4-8ac2-bf4e042b62c8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:49.556995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea6d65-cc02-45c4-8ac2-bf4e042b62c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:49.556995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea6d65-cc02-45c4-8ac2-bf4e042b62c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:49.563367 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:49.567596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea6d65-cc02-45c4-8ac2-bf4e042b62c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-45ea6d65-cc02-45c4-8ac2-bf4e042b62c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:46:49.572404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:49.573600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:49.574606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 469/926] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 12:46:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:49.579686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af6f91-44c4-4d60-ad13-262b45a91c45 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:49.634341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:49.679506 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af6f91-44c4-4d60-ad13-262b45a91c45 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:49.715207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73338f-3cc6-41bf-8222-88a822453174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:49.715890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73338f-3cc6-41bf-8222-88a822453174 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:49.716223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73338f-3cc6-41bf-8222-88a822453174 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:49.716493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73338f-3cc6-41bf-8222-88a822453174 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:49.717139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73338f-3cc6-41bf-8222-88a822453174 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:49.717489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73338f-3cc6-41bf-8222-88a822453174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:49.718857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73338f-3cc6-41bf-8222-88a822453174 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:49.726797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 458/927] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 12:46:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:49.732750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea6d65-cc02-45c4-8ac2-bf4e042b62c8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:49.835356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea6d65-cc02-45c4-8ac2-bf4e042b62c8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:49.868591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ee0b5-0341-4fdd-9ede-f131e9076b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:49.869237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ee0b5-0341-4fdd-9ede-f131e9076b60 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:49.869521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ee0b5-0341-4fdd-9ede-f131e9076b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:49.869795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ee0b5-0341-4fdd-9ede-f131e9076b60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:49.880277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ee0b5-0341-4fdd-9ede-f131e9076b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:49.881884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ee0b5-0341-4fdd-9ede-f131e9076b60 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2f7ee0b5-0341-4fdd-9ede-f131e9076b60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:46:49.896697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:49.897809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:46:49.898852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:46:49.898852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:49.898852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:49.898852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 470/928] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:49.903668 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73338f-3cc6-41bf-8222-88a822453174 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:50.000590 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73338f-3cc6-41bf-8222-88a822453174 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:50.016619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:50.034005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cf9ff-3683-4cee-bf1f-99e43552f6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:50.034696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cf9ff-3683-4cee-bf1f-99e43552f6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:50.035104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cf9ff-3683-4cee-bf1f-99e43552f6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:50.035382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cf9ff-3683-4cee-bf1f-99e43552f6c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:50.036059 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cf9ff-3683-4cee-bf1f-99e43552f6c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:50.036364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cf9ff-3683-4cee-bf1f-99e43552f6c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:50.037751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cf9ff-3683-4cee-bf1f-99e43552f6c3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:50.049060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 459/929] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:50.055310 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ee0b5-0341-4fdd-9ede-f131e9076b60 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:50.104267 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:50.152564 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ee0b5-0341-4fdd-9ede-f131e9076b60 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:50.184947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46ab3-0177-4fea-9aac-fb9f8b2c1753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:50.185591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46ab3-0177-4fea-9aac-fb9f8b2c1753 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:50.185878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46ab3-0177-4fea-9aac-fb9f8b2c1753 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:50.186149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46ab3-0177-4fea-9aac-fb9f8b2c1753 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:50.194059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f46ab3-0177-4fea-9aac-fb9f8b2c1753 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:50.195486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f46ab3-0177-4fea-9aac-fb9f8b2c1753 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-59f46ab3-0177-4fea-9aac-fb9f8b2c1753 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:46:50.206289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:50.210249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:50.211188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 471/930] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:50.213535 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cf9ff-3683-4cee-bf1f-99e43552f6c3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:50.310753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cf9ff-3683-4cee-bf1f-99e43552f6c3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:50.345606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fb108-180e-465e-9506-c4dfb7c58106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:50.346207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fb108-180e-465e-9506-c4dfb7c58106 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:50.346489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fb108-180e-465e-9506-c4dfb7c58106 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:50.346766 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fb108-180e-465e-9506-c4dfb7c58106 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:50.347487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fb108-180e-465e-9506-c4dfb7c58106 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:50.347941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fb108-180e-465e-9506-c4dfb7c58106 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:50.349387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fb108-180e-465e-9506-c4dfb7c58106 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:50.366621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 460/931] 10.222.0.30 () {56 vars in 1143 bytes} [Sat Oct 15 12:46:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:50.372698 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46ab3-0177-4fea-9aac-fb9f8b2c1753 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:50.471102 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46ab3-0177-4fea-9aac-fb9f8b2c1753 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:50.481598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:50.504514 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf1e2d-5ad8-44b1-b3b1-2e4b80d5a2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:50.505118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf1e2d-5ad8-44b1-b3b1-2e4b80d5a2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:50.505429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf1e2d-5ad8-44b1-b3b1-2e4b80d5a2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:50.505705 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf1e2d-5ad8-44b1-b3b1-2e4b80d5a2d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:50.571412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf1e2d-5ad8-44b1-b3b1-2e4b80d5a2d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:50.572886 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf1e2d-5ad8-44b1-b3b1-2e4b80d5a2d5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:50.671823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 472/932] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:46:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 12:46:50.678502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fb108-180e-465e-9506-c4dfb7c58106 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:50.711415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:50.776168 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fb108-180e-465e-9506-c4dfb7c58106 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:50.809786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41938b-21d3-421e-a936-bf682def0eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:50.810419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41938b-21d3-421e-a936-bf682def0eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:50.810697 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41938b-21d3-421e-a936-bf682def0eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:50.810968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41938b-21d3-421e-a936-bf682def0eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:50.820610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41938b-21d3-421e-a936-bf682def0eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:50.822172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41938b-21d3-421e-a936-bf682def0eb9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5e41938b-21d3-421e-a936-bf682def0eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:46:50.836773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:50.837909 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:46:50.839139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:46:50.839139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:50.839139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:50.839139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 461/933] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:50.843677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf1e2d-5ad8-44b1-b3b1-2e4b80d5a2d5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:50.855595 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:50.939915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf1e2d-5ad8-44b1-b3b1-2e4b80d5a2d5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:50.972931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2721ef-1e21-4d6c-9647-87700646bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:50.973560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2721ef-1e21-4d6c-9647-87700646bb01 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:50.973843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2721ef-1e21-4d6c-9647-87700646bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:50.974120 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2721ef-1e21-4d6c-9647-87700646bb01 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:50.974924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2721ef-1e21-4d6c-9647-87700646bb01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:50.975231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2721ef-1e21-4d6c-9647-87700646bb01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:50.976605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2721ef-1e21-4d6c-9647-87700646bb01 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:50.987371 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 473/934] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:50.993112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41938b-21d3-421e-a936-bf682def0eb9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:51.077598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:51.093555 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41938b-21d3-421e-a936-bf682def0eb9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:51.126074 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044524-e3b7-4c6a-892e-e0117487ef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:51.126729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044524-e3b7-4c6a-892e-e0117487ef1f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:51.127080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044524-e3b7-4c6a-892e-e0117487ef1f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:51.127353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044524-e3b7-4c6a-892e-e0117487ef1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:51.133770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2044524-e3b7-4c6a-892e-e0117487ef1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:51.135187 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2044524-e3b7-4c6a-892e-e0117487ef1f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f2044524-e3b7-4c6a-892e-e0117487ef1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:46:51.146979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:51.148091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:51.149038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:46:51.149038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:46:51.149038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:46:51.149038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:46:51.149038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:46:51.149038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:46:51.149038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:46:51.149038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:51.149038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 462/935] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:51.154169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2721ef-1e21-4d6c-9647-87700646bb01 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:51.223497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:51.252279 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2721ef-1e21-4d6c-9647-87700646bb01 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:51.285772 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6d7a9-116e-4917-846e-c8f7ad95c3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:51.286368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d7a9-116e-4917-846e-c8f7ad95c3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:51.286646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d7a9-116e-4917-846e-c8f7ad95c3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:51.286915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d7a9-116e-4917-846e-c8f7ad95c3b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:51.287587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d7a9-116e-4917-846e-c8f7ad95c3b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:51.287889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d7a9-116e-4917-846e-c8f7ad95c3b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:51.289595 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d7a9-116e-4917-846e-c8f7ad95c3b9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:51.301175 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 474/936] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:46:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:51.307385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044524-e3b7-4c6a-892e-e0117487ef1f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:51.407532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044524-e3b7-4c6a-892e-e0117487ef1f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:51.440044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb0060-add1-4b72-9ba3-554c4f560fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:51.440646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0060-add1-4b72-9ba3-554c4f560fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:51.441498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0060-add1-4b72-9ba3-554c4f560fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:51.441727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0060-add1-4b72-9ba3-554c4f560fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:51.442442 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb0060-add1-4b72-9ba3-554c4f560fa6 None admin] RBAC: Adding query filter params (role.id=5a5e7aced6d84267bd9e1fa0c1016420, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=18df1f401bb2494299eda9582c85a04f) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:51.442751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb0060-add1-4b72-9ba3-554c4f560fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:51.444181 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb0060-add1-4b72-9ba3-554c4f560fa6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:51.449782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 463/937] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:46:51 2022] GET /identity/v3/role_assignments?user.id=18df1f401bb2494299eda9582c85a04f&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=5a5e7aced6d84267bd9e1fa0c1016420 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:53.604259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a85f44-7488-4e3b-9f75-07d1eeaf1b59 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:53.604814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a85f44-7488-4e3b-9f75-07d1eeaf1b59 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:53.604928 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a85f44-7488-4e3b-9f75-07d1eeaf1b59 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:53.606252 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 475/938] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:46:53.611460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dc3f4-6f1f-4087-a26d-c994b8e993ea None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:53.611860 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dc3f4-6f1f-4087-a26d-c994b8e993ea None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:53.612016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dc3f4-6f1f-4087-a26d-c994b8e993ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:53.631446 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e32dc3f4-6f1f-4087-a26d-c994b8e993ea None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:46:53.632623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e32dc3f4-6f1f-4087-a26d-c994b8e993ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:46:53.635335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:53.697262 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 464/939] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:46:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:46:53.970596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a85f44-7488-4e3b-9f75-07d1eeaf1b59 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:54.069494 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a85f44-7488-4e3b-9f75-07d1eeaf1b59 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:54.096391 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:54.102174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439384e-f2c3-41e3-9ffd-5f092ab3fc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:54.102828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439384e-f2c3-41e3-9ffd-5f092ab3fc8f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:54.103105 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439384e-f2c3-41e3-9ffd-5f092ab3fc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:54.103376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439384e-f2c3-41e3-9ffd-5f092ab3fc8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:54.113508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439384e-f2c3-41e3-9ffd-5f092ab3fc8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b439384e-f2c3-41e3-9ffd-5f092ab3fc8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:46:54.118200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:54.119174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:54.120085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 476/940] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:46:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:54.125656 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dc3f4-6f1f-4087-a26d-c994b8e993ea None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:54.225117 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dc3f4-6f1f-4087-a26d-c994b8e993ea None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:54.258516 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee28a29-2996-4f8e-927f-37b49c026949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:54.259113 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee28a29-2996-4f8e-927f-37b49c026949 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:54.259389 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee28a29-2996-4f8e-927f-37b49c026949 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:54.259657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee28a29-2996-4f8e-927f-37b49c026949 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:54.260305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee28a29-2996-4f8e-927f-37b49c026949 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:54.260602 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee28a29-2996-4f8e-927f-37b49c026949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:54.261974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee28a29-2996-4f8e-927f-37b49c026949 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:54.269647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 465/941] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:46:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:54.275530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439384e-f2c3-41e3-9ffd-5f092ab3fc8f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:54.373445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439384e-f2c3-41e3-9ffd-5f092ab3fc8f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:54.406656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db92fce-6f5c-47b3-897f-1568cc20335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:54.407282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db92fce-6f5c-47b3-897f-1568cc20335d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:54.407565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db92fce-6f5c-47b3-897f-1568cc20335d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:54.407840 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db92fce-6f5c-47b3-897f-1568cc20335d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:54.418736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db92fce-6f5c-47b3-897f-1568cc20335d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:54.420195 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db92fce-6f5c-47b3-897f-1568cc20335d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6db92fce-6f5c-47b3-897f-1568cc20335d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:46:54.435538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:54.436631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:46:54.437627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:46:54.437627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:54.437627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:54.437627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 477/942] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:54.442515 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee28a29-2996-4f8e-927f-37b49c026949 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:54.445464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:54.542191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee28a29-2996-4f8e-927f-37b49c026949 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:54.575868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b3a3-5ed6-455a-a61f-8d92e0b9747a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:54.576485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b3a3-5ed6-455a-a61f-8d92e0b9747a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:54.576762 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b3a3-5ed6-455a-a61f-8d92e0b9747a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:54.577040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b3a3-5ed6-455a-a61f-8d92e0b9747a None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:54.577707 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069b3a3-5ed6-455a-a61f-8d92e0b9747a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:54.578012 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069b3a3-5ed6-455a-a61f-8d92e0b9747a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:54.579370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069b3a3-5ed6-455a-a61f-8d92e0b9747a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:54.591201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 466/943] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:54.598172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db92fce-6f5c-47b3-897f-1568cc20335d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:54.640393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:54.697147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db92fce-6f5c-47b3-897f-1568cc20335d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:54.729286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0931e72-a097-40bf-9e2d-0ef68b487eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:54.729925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0931e72-a097-40bf-9e2d-0ef68b487eac None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:54.730185 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0931e72-a097-40bf-9e2d-0ef68b487eac None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:54.730457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0931e72-a097-40bf-9e2d-0ef68b487eac None admin] PATH_INFO: `/v3/users/placement` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:54.738127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0931e72-a097-40bf-9e2d-0ef68b487eac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:54.739566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0931e72-a097-40bf-9e2d-0ef68b487eac None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f0931e72-a097-40bf-9e2d-0ef68b487eac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:46:54.750457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:54.751565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:54.752549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 478/944] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:54.757947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b3a3-5ed6-455a-a61f-8d92e0b9747a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:54.818711 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:54.854899 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b3a3-5ed6-455a-a61f-8d92e0b9747a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:54.888342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c56fe-e120-4e88-8704-b085e4b32d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:54.888939 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c56fe-e120-4e88-8704-b085e4b32d95 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:54.889237 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c56fe-e120-4e88-8704-b085e4b32d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:54.889515 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c56fe-e120-4e88-8704-b085e4b32d95 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:54.890281 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c56fe-e120-4e88-8704-b085e4b32d95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:54.890768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c56fe-e120-4e88-8704-b085e4b32d95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:54.892463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c56fe-e120-4e88-8704-b085e4b32d95 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:54.910236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 467/945] 10.222.0.30 () {56 vars in 1143 bytes} [Sat Oct 15 12:46:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:54.916429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0931e72-a097-40bf-9e2d-0ef68b487eac None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:55.014062 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0931e72-a097-40bf-9e2d-0ef68b487eac None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:55.015550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:55.047644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee81ac1-bb87-45c7-b63d-579724e0f773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:55.048291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81ac1-bb87-45c7-b63d-579724e0f773 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:55.048572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81ac1-bb87-45c7-b63d-579724e0f773 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:55.048847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81ac1-bb87-45c7-b63d-579724e0f773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:55.058940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee81ac1-bb87-45c7-b63d-579724e0f773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:55.060386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee81ac1-bb87-45c7-b63d-579724e0f773 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4ee81ac1-bb87-45c7-b63d-579724e0f773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:46:55.076953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:55.078133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:46:55.079275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:46:55.079275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:55.079275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:55.079275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 479/946] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:55.082066 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c56fe-e120-4e88-8704-b085e4b32d95 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:55.179756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c56fe-e120-4e88-8704-b085e4b32d95 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:55.199522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:55.213026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a853f9-f4d1-48c4-8d68-6279ef8e8168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:55.213683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a853f9-f4d1-48c4-8d68-6279ef8e8168 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:55.213993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a853f9-f4d1-48c4-8d68-6279ef8e8168 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:55.214268 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a853f9-f4d1-48c4-8d68-6279ef8e8168 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:55.214905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a853f9-f4d1-48c4-8d68-6279ef8e8168 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:55.215210 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a853f9-f4d1-48c4-8d68-6279ef8e8168 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:55.216568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a853f9-f4d1-48c4-8d68-6279ef8e8168 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:55.227956 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 468/947] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:55.233808 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee81ac1-bb87-45c7-b63d-579724e0f773 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:55.332083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee81ac1-bb87-45c7-b63d-579724e0f773 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:55.365055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977079a-ab23-48a6-9b60-bc6d4c35f405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:55.365726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977079a-ab23-48a6-9b60-bc6d4c35f405 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:55.365994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977079a-ab23-48a6-9b60-bc6d4c35f405 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:55.366271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977079a-ab23-48a6-9b60-bc6d4c35f405 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:55.372773 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977079a-ab23-48a6-9b60-bc6d4c35f405 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:55.374294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977079a-ab23-48a6-9b60-bc6d4c35f405 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6977079a-ab23-48a6-9b60-bc6d4c35f405 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:46:55.385495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:55.386435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:55.387375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:46:55.387375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:46:55.387375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:46:55.387375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:46:55.387375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:46:55.387375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:46:55.387375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:46:55.387375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:55.387375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 480/948] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:55.393046 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a853f9-f4d1-48c4-8d68-6279ef8e8168 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:55.490705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a853f9-f4d1-48c4-8d68-6279ef8e8168 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:55.523253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b934f3e-d853-4683-9511-14434c56a587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:55.523843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b934f3e-d853-4683-9511-14434c56a587 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:55.524120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b934f3e-d853-4683-9511-14434c56a587 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:55.524434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b934f3e-d853-4683-9511-14434c56a587 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:55.525157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b934f3e-d853-4683-9511-14434c56a587 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:55.525495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b934f3e-d853-4683-9511-14434c56a587 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:55.526901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b934f3e-d853-4683-9511-14434c56a587 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:55.540064 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 469/949] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:55.546149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977079a-ab23-48a6-9b60-bc6d4c35f405 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:55.562650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:55.647328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977079a-ab23-48a6-9b60-bc6d4c35f405 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:55.682261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ccd2-3ad1-4321-a92d-6b326bcb6d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:55.682923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ccd2-3ad1-4321-a92d-6b326bcb6d54 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:55.683207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ccd2-3ad1-4321-a92d-6b326bcb6d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:55.683485 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ccd2-3ad1-4321-a92d-6b326bcb6d54 None admin] PATH_INFO: `/v3/projects/3c6d59e727614128a4304872ad0867cd/users/18df1f401bb2494299eda9582c85a04f/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:55.685394 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9ccd2-3ad1-4321-a92d-6b326bcb6d54 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c6d59e727614128a4304872ad0867cd, user_id=18df1f401bb2494299eda9582c85a04f, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:55.686831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9ccd2-3ad1-4321-a92d-6b326bcb6d54 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:55.788045 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 481/950] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 12:46:55 2022] PUT /identity/v3/projects/3c6d59e727614128a4304872ad0867cd/users/18df1f401bb2494299eda9582c85a04f/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 12:46:57.934592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422188ed-424e-4fb0-9f20-3beb23775561 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:57.935107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422188ed-424e-4fb0-9f20-3beb23775561 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:57.935107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422188ed-424e-4fb0-9f20-3beb23775561 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:57.936503 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 470/951] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:46:57.941741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80f9f8-5284-45f5-aa37-760ea28552e3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:57.942161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80f9f8-5284-45f5-aa37-760ea28552e3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:57.942303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80f9f8-5284-45f5-aa37-760ea28552e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:57.960135 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-be80f9f8-5284-45f5-aa37-760ea28552e3 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:46:57.961368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-be80f9f8-5284-45f5-aa37-760ea28552e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:46:58.027995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 482/952] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:46:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:46:58.299001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422188ed-424e-4fb0-9f20-3beb23775561 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:58.342295 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:58.399009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422188ed-424e-4fb0-9f20-3beb23775561 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:58.432793 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f82efa-6dab-419b-846a-5bbfa2f74021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:58.433497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f82efa-6dab-419b-846a-5bbfa2f74021 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:58.433774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f82efa-6dab-419b-846a-5bbfa2f74021 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:58.434052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f82efa-6dab-419b-846a-5bbfa2f74021 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:58.444504 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f82efa-6dab-419b-846a-5bbfa2f74021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c3f82efa-6dab-419b-846a-5bbfa2f74021 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 12:46:58.449247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:58.450914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:58.451958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 471/953] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 12:46:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:58.456543 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80f9f8-5284-45f5-aa37-760ea28552e3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:58.459694 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:58.551352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80f9f8-5284-45f5-aa37-760ea28552e3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:58.585985 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78ae09-fd18-4bc9-9269-24788fa5bfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:58.586586 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78ae09-fd18-4bc9-9269-24788fa5bfa3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:58.587013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78ae09-fd18-4bc9-9269-24788fa5bfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:58.587285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78ae09-fd18-4bc9-9269-24788fa5bfa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:58.587940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78ae09-fd18-4bc9-9269-24788fa5bfa3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:58.588239 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78ae09-fd18-4bc9-9269-24788fa5bfa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:58.589644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78ae09-fd18-4bc9-9269-24788fa5bfa3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:58.597921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 483/954] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 12:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:58.603873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f82efa-6dab-419b-846a-5bbfa2f74021 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:58.701691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:58.704405 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f82efa-6dab-419b-846a-5bbfa2f74021 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:58.736975 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301822a4-e018-44df-898e-f0113bcb0806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:58.738242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301822a4-e018-44df-898e-f0113bcb0806 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:58.738526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301822a4-e018-44df-898e-f0113bcb0806 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:58.738803 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301822a4-e018-44df-898e-f0113bcb0806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:58.749133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301822a4-e018-44df-898e-f0113bcb0806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:58.750662 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301822a4-e018-44df-898e-f0113bcb0806 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-301822a4-e018-44df-898e-f0113bcb0806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:46:58.764714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:58.765907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:46:58.767016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:46:58.767016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:58.767016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:58.767016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 472/955] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:58.771806 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78ae09-fd18-4bc9-9269-24788fa5bfa3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:58.837261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:58.873041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78ae09-fd18-4bc9-9269-24788fa5bfa3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:58.906345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670e6d5-0831-4a90-85af-340defa07797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:58.911078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670e6d5-0831-4a90-85af-340defa07797 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:58.911078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670e6d5-0831-4a90-85af-340defa07797 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:58.911078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670e6d5-0831-4a90-85af-340defa07797 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:58.911078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670e6d5-0831-4a90-85af-340defa07797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:58.911078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670e6d5-0831-4a90-85af-340defa07797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:58.911078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670e6d5-0831-4a90-85af-340defa07797 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:58.922235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 484/956] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:58.928947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301822a4-e018-44df-898e-f0113bcb0806 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:59.027887 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301822a4-e018-44df-898e-f0113bcb0806 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:59.060443 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9673c-5c68-41c1-95ac-aaa0245e762c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:59.061052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9673c-5c68-41c1-95ac-aaa0245e762c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:59.061357 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9673c-5c68-41c1-95ac-aaa0245e762c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:59.061628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9673c-5c68-41c1-95ac-aaa0245e762c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:59.068722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9673c-5c68-41c1-95ac-aaa0245e762c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:59.070170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9673c-5c68-41c1-95ac-aaa0245e762c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1da9673c-5c68-41c1-95ac-aaa0245e762c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 12:46:59.080915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:59.082694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 12:46:59.083653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:46:59.084357 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 473/957] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:59.088682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670e6d5-0831-4a90-85af-340defa07797 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:59.186496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670e6d5-0831-4a90-85af-340defa07797 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:59.211702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:59.219795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bec79-5f72-4504-9929-ec84a811bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:59.220391 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bec79-5f72-4504-9929-ec84a811bd4c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:59.220667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bec79-5f72-4504-9929-ec84a811bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:59.220936 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bec79-5f72-4504-9929-ec84a811bd4c None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:59.221751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bec79-5f72-4504-9929-ec84a811bd4c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:59.222272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bec79-5f72-4504-9929-ec84a811bd4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:59.223698 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bec79-5f72-4504-9929-ec84a811bd4c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:59.240399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 485/958] 10.222.0.30 () {56 vars in 1143 bytes} [Sat Oct 15 12:46:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:59.246140 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9673c-5c68-41c1-95ac-aaa0245e762c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:59.261649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:59.342982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9673c-5c68-41c1-95ac-aaa0245e762c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:59.374679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76780b-068b-4869-bfe6-1edbe72d82fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:59.375351 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76780b-068b-4869-bfe6-1edbe72d82fa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:59.375595 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76780b-068b-4869-bfe6-1edbe72d82fa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:59.375871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76780b-068b-4869-bfe6-1edbe72d82fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:59.441149 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76780b-068b-4869-bfe6-1edbe72d82fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:59.442624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76780b-068b-4869-bfe6-1edbe72d82fa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:59.481352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:59.540928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 474/959] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 12:46:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 12:46:59.546818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bec79-5f72-4504-9929-ec84a811bd4c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:59.649030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bec79-5f72-4504-9929-ec84a811bd4c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:59.682280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba5a28-4dff-49a0-befb-e73ce2945777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:59.682906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba5a28-4dff-49a0-befb-e73ce2945777 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:59.683189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba5a28-4dff-49a0-befb-e73ce2945777 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:59.683466 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba5a28-4dff-49a0-befb-e73ce2945777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:59.694490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba5a28-4dff-49a0-befb-e73ce2945777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:59.695975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba5a28-4dff-49a0-befb-e73ce2945777 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4aba5a28-4dff-49a0-befb-e73ce2945777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 12:46:59.710973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:46:59.712032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 12:46:59.712930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 12:46:59.712930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 12:46:59.712930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:46:59.712930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 486/960] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:46:59.717920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76780b-068b-4869-bfe6-1edbe72d82fa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:59.814802 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76780b-068b-4869-bfe6-1edbe72d82fa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:46:59.827980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:59.848105 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f433cd-47a4-49ae-957b-0f904c57725e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:46:59.848706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f433cd-47a4-49ae-957b-0f904c57725e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:46:59.848978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f433cd-47a4-49ae-957b-0f904c57725e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:46:59.850226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f433cd-47a4-49ae-957b-0f904c57725e None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:46:59.850226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f433cd-47a4-49ae-957b-0f904c57725e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:46:59.850226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f433cd-47a4-49ae-957b-0f904c57725e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:46:59.851533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f433cd-47a4-49ae-957b-0f904c57725e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:46:59.864993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 475/961] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 12:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:46:59.870382 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba5a28-4dff-49a0-befb-e73ce2945777 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:46:59.904975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:46:59.966988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba5a28-4dff-49a0-befb-e73ce2945777 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:00.000205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cd6dd-1a87-464a-a01c-9dcab2f80ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:00.000828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cd6dd-1a87-464a-a01c-9dcab2f80ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:00.001104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cd6dd-1a87-464a-a01c-9dcab2f80ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:00.001405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cd6dd-1a87-464a-a01c-9dcab2f80ae0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:00.008529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0cd6dd-1a87-464a-a01c-9dcab2f80ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:00.009991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0cd6dd-1a87-464a-a01c-9dcab2f80ae0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6b0cd6dd-1a87-464a-a01c-9dcab2f80ae0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 12:47:00.020759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:47:00.021832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:47:00.022770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 12:47:00.022770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 12:47:00.022770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 12:47:00.022770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 12:47:00.022770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 12:47:00.022770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 12:47:00.022770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 12:47:00.022770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 15 12:47:00.022770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 487/962] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 12:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 12:47:00.027855 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f433cd-47a4-49ae-957b-0f904c57725e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:00.126922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f433cd-47a4-49ae-957b-0f904c57725e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:00.160609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa4c57-5a1d-4ab3-9dce-c0e8c0f1d7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:00.161150 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa4c57-5a1d-4ab3-9dce-c0e8c0f1d7dd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:00.161460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa4c57-5a1d-4ab3-9dce-c0e8c0f1d7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:00.161734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa4c57-5a1d-4ab3-9dce-c0e8c0f1d7dd None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:00.162409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa4c57-5a1d-4ab3-9dce-c0e8c0f1d7dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:47:00.162716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa4c57-5a1d-4ab3-9dce-c0e8c0f1d7dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:00.164395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa4c57-5a1d-4ab3-9dce-c0e8c0f1d7dd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:00.176757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 476/963] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 12:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:47:00.182918 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cd6dd-1a87-464a-a01c-9dcab2f80ae0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:00.273411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:00.279048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cd6dd-1a87-464a-a01c-9dcab2f80ae0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:00.312678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe017d9c-c3e0-4324-af9b-1da609307baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:00.313297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe017d9c-c3e0-4324-af9b-1da609307baa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:00.313578 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe017d9c-c3e0-4324-af9b-1da609307baa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:00.313849 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe017d9c-c3e0-4324-af9b-1da609307baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:00.314559 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe017d9c-c3e0-4324-af9b-1da609307baa None admin] RBAC: Adding query filter params (role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5, scope.project.id=3c6d59e727614128a4304872ad0867cd, user.id=18df1f401bb2494299eda9582c85a04f) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:47:00.314863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe017d9c-c3e0-4324-af9b-1da609307baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:00.316333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe017d9c-c3e0-4324-af9b-1da609307baa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:00.321943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 488/964] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 12:47:00 2022] GET /identity/v3/role_assignments?user.id=18df1f401bb2494299eda9582c85a04f&scope.project.id=3c6d59e727614128a4304872ad0867cd&role.id=d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:47:02.467310 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fda71d-f62e-458f-a6bf-fd2dbaf815c9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:02.467864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fda71d-f62e-458f-a6bf-fd2dbaf815c9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:02.467864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fda71d-f62e-458f-a6bf-fd2dbaf815c9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:02.469214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 477/965] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:47:02.474555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe37375-1b1e-4f1d-839b-baaee65a86c9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:02.474962 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe37375-1b1e-4f1d-839b-baaee65a86c9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:02.475107 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe37375-1b1e-4f1d-839b-baaee65a86c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:02.493539 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1fe37375-1b1e-4f1d-839b-baaee65a86c9 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:47:02.494771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1fe37375-1b1e-4f1d-839b-baaee65a86c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:47:02.560851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 489/966] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:47:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:47:02.826625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fda71d-f62e-458f-a6bf-fd2dbaf815c9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:02.857627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:02.927267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fda71d-f62e-458f-a6bf-fd2dbaf815c9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:02.961005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763cff5-0871-4e9d-a803-a702096311e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:02.961670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763cff5-0871-4e9d-a803-a702096311e1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:02.961939 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763cff5-0871-4e9d-a803-a702096311e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:02.962214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763cff5-0871-4e9d-a803-a702096311e1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:02.962790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763cff5-0871-4e9d-a803-a702096311e1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:02.964143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763cff5-0871-4e9d-a803-a702096311e1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b763cff5-0871-4e9d-a803-a702096311e1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:47:02.972863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:47:02.974160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:47:02.975034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 478/967] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 12:47:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:47:02.979978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe37375-1b1e-4f1d-839b-baaee65a86c9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:03.070463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:03.079778 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe37375-1b1e-4f1d-839b-baaee65a86c9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:03.112307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de94078-f90c-40dd-b93c-0ce6379689d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:03.113428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de94078-f90c-40dd-b93c-0ce6379689d6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:03.113696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de94078-f90c-40dd-b93c-0ce6379689d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:03.113963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de94078-f90c-40dd-b93c-0ce6379689d6 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:03.114576 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de94078-f90c-40dd-b93c-0ce6379689d6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:47:03.114874 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de94078-f90c-40dd-b93c-0ce6379689d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:03.116233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de94078-f90c-40dd-b93c-0ce6379689d6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:03.120918 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 490/968] 10.222.0.30 () {56 vars in 1116 bytes} [Sat Oct 15 12:47:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:47:03.126863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763cff5-0871-4e9d-a803-a702096311e1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:03.218744 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:03.224797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763cff5-0871-4e9d-a803-a702096311e1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:03.256703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c89ebe-2860-4c7b-891b-50d8d0b8b93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:03.257328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c89ebe-2860-4c7b-891b-50d8d0b8b93b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:03.257607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c89ebe-2860-4c7b-891b-50d8d0b8b93b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:03.257878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c89ebe-2860-4c7b-891b-50d8d0b8b93b None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:03.258496 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c89ebe-2860-4c7b-891b-50d8d0b8b93b None admin] RBAC: Adding query filter params (type=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:47:03.258796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c89ebe-2860-4c7b-891b-50d8d0b8b93b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:03.260170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c89ebe-2860-4c7b-891b-50d8d0b8b93b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:03.264433 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 479/969] 10.222.0.30 () {56 vars in 1116 bytes} [Sat Oct 15 12:47:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:47:05.378583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae19d6-1ca4-4adb-bbed-890ebe46b6fc None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:05.379148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae19d6-1ca4-4adb-bbed-890ebe46b6fc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:05.379148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae19d6-1ca4-4adb-bbed-890ebe46b6fc None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:05.380505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 491/970] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:47:05.385684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffaa864-f084-4419-8932-9eafc18ea345 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:05.386100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffaa864-f084-4419-8932-9eafc18ea345 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:05.386236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffaa864-f084-4419-8932-9eafc18ea345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:05.405042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fffaa864-f084-4419-8932-9eafc18ea345 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:47:05.406263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fffaa864-f084-4419-8932-9eafc18ea345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:47:05.473267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 480/971] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:47:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:47:05.743896 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae19d6-1ca4-4adb-bbed-890ebe46b6fc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:05.843334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae19d6-1ca4-4adb-bbed-890ebe46b6fc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:05.884161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019a7da-e345-4241-aab8-f3536b130eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:05.884856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019a7da-e345-4241-aab8-f3536b130eed None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:05.885100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019a7da-e345-4241-aab8-f3536b130eed None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:05.885403 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019a7da-e345-4241-aab8-f3536b130eed None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:05.886062 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019a7da-e345-4241-aab8-f3536b130eed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:05.887435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019a7da-e345-4241-aab8-f3536b130eed None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:05.976870 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 492/972] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 12:47:05 2022] POST /identity/v3/services => generated 246 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 12:47:08.137625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeeb07-9bd9-4b10-b0c7-fcc858f8c56b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:08.138181 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeeb07-9bd9-4b10-b0c7-fcc858f8c56b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:08.138181 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeeb07-9bd9-4b10-b0c7-fcc858f8c56b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:08.139511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 481/973] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:47:08.145253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b8f08-e790-4303-8770-9f4a4c3d9884 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:08.145663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b8f08-e790-4303-8770-9f4a4c3d9884 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:08.145811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b8f08-e790-4303-8770-9f4a4c3d9884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:08.163790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-cd3b8f08-e790-4303-8770-9f4a4c3d9884 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:47:08.165036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cd3b8f08-e790-4303-8770-9f4a4c3d9884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:47:08.174629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:08.237150 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 493/974] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:47:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:47:08.507219 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeeeb07-9bd9-4b10-b0c7-fcc858f8c56b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:08.598822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:08.609415 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeeeb07-9bd9-4b10-b0c7-fcc858f8c56b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:08.642025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3083c-59f6-4a09-bf88-b6db719f85ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:08.642646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3083c-59f6-4a09-bf88-b6db719f85ff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:08.642921 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3083c-59f6-4a09-bf88-b6db719f85ff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:08.643191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3083c-59f6-4a09-bf88-b6db719f85ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:08.643786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3083c-59f6-4a09-bf88-b6db719f85ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:08.645139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3083c-59f6-4a09-bf88-b6db719f85ff None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-34f3083c-59f6-4a09-bf88-b6db719f85ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:47:08.653679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:47:08.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:47:08.656016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 482/975] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 12:47:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:47:08.664149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b8f08-e790-4303-8770-9f4a4c3d9884 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:08.762325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b8f08-e790-4303-8770-9f4a4c3d9884 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:08.796701 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73ead1-328b-40b2-87d4-c6d4925ec1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:08.797343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73ead1-328b-40b2-87d4-c6d4925ec1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:08.797626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73ead1-328b-40b2-87d4-c6d4925ec1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:08.797899 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73ead1-328b-40b2-87d4-c6d4925ec1a5 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:08.798551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73ead1-328b-40b2-87d4-c6d4925ec1a5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:47:08.798853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73ead1-328b-40b2-87d4-c6d4925ec1a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:08.800270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73ead1-328b-40b2-87d4-c6d4925ec1a5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:08.805445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 494/976] 10.222.0.30 () {56 vars in 1116 bytes} [Sat Oct 15 12:47:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 12:47:08.811439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3083c-59f6-4a09-bf88-b6db719f85ff None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:08.910294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3083c-59f6-4a09-bf88-b6db719f85ff None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:08.944142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05b612-abb5-45d2-81e9-6e0cb6673612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:08.944760 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05b612-abb5-45d2-81e9-6e0cb6673612 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:08.945042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05b612-abb5-45d2-81e9-6e0cb6673612 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:08.945373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05b612-abb5-45d2-81e9-6e0cb6673612 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:08.946118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05b612-abb5-45d2-81e9-6e0cb6673612 None admin] RBAC: Adding query filter params (interface=public, service_id=7d76a755cf8e4a7bae743f8a5ac3cb4a, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:47:08.946430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05b612-abb5-45d2-81e9-6e0cb6673612 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:08.947895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05b612-abb5-45d2-81e9-6e0cb6673612 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:08.953729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 483/977] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 12:47:08 2022] GET /identity/v3/endpoints?service_id=7d76a755cf8e4a7bae743f8a5ac3cb4a&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:47:08.960597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73ead1-328b-40b2-87d4-c6d4925ec1a5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:08.964593 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:09.058668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73ead1-328b-40b2-87d4-c6d4925ec1a5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:09.093393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b4964-115a-43e6-9a0d-ef469275dfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:09.093997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b4964-115a-43e6-9a0d-ef469275dfd3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:09.094274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b4964-115a-43e6-9a0d-ef469275dfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:09.094546 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b4964-115a-43e6-9a0d-ef469275dfd3 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:09.095145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b4964-115a-43e6-9a0d-ef469275dfd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:09.096508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b4964-115a-43e6-9a0d-ef469275dfd3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:09.103819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 495/978] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 12:47:08 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 12:47:11.322626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a037-7cda-464a-ad16-e445c37b03c5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:11.323229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a037-7cda-464a-ad16-e445c37b03c5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:11.323332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a037-7cda-464a-ad16-e445c37b03c5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:11.324594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 484/979] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:47:11.329760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165ac2f-ddab-4a96-9816-715ecd711b19 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:11.330535 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165ac2f-ddab-4a96-9816-715ecd711b19 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:11.330535 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165ac2f-ddab-4a96-9816-715ecd711b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:11.348497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0165ac2f-ddab-4a96-9816-715ecd711b19 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:47:11.349727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0165ac2f-ddab-4a96-9816-715ecd711b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:47:11.410429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:11.416044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 496/980] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:47:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 12:47:11.682316 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327a037-7cda-464a-ad16-e445c37b03c5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:11.702279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:11.782485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327a037-7cda-464a-ad16-e445c37b03c5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:11.816042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b9f7b-fdc4-4b54-9305-592a7a3cfb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:11.816672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b9f7b-fdc4-4b54-9305-592a7a3cfb31 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:11.816947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b9f7b-fdc4-4b54-9305-592a7a3cfb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:11.817243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b9f7b-fdc4-4b54-9305-592a7a3cfb31 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:11.817823 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b9f7b-fdc4-4b54-9305-592a7a3cfb31 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:11.819173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b9f7b-fdc4-4b54-9305-592a7a3cfb31 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-2a4b9f7b-fdc4-4b54-9305-592a7a3cfb31 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 12:47:11.827757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 12:47:11.831242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Oct 15 12:47:11.832477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 485/981] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 12:47:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 12:47:11.834991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165ac2f-ddab-4a96-9816-715ecd711b19 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:11.937605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165ac2f-ddab-4a96-9816-715ecd711b19 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:11.970365 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244a47-610c-4fa4-b465-8d3f553a5b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:11.970962 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244a47-610c-4fa4-b465-8d3f553a5b7c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:11.971243 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244a47-610c-4fa4-b465-8d3f553a5b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:11.971565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244a47-610c-4fa4-b465-8d3f553a5b7c None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:11.972428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244a47-610c-4fa4-b465-8d3f553a5b7c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 12:47:11.972733 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244a47-610c-4fa4-b465-8d3f553a5b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:11.974136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244a47-610c-4fa4-b465-8d3f553a5b7c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:11.981533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 497/982] 10.222.0.30 () {56 vars in 1116 bytes} [Sat Oct 15 12:47:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 12:47:11.985737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b9f7b-fdc4-4b54-9305-592a7a3cfb31 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:47:12.066899 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:12.084077 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b9f7b-fdc4-4b54-9305-592a7a3cfb31 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:47:12.117112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e928-6d8f-4da4-9671-73024ad5736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:47:12.117764 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e928-6d8f-4da4-9671-73024ad5736d None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:47:12.118040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e928-6d8f-4da4-9671-73024ad5736d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:47:12.118308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e928-6d8f-4da4-9671-73024ad5736d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:47:12.119008 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e928-6d8f-4da4-9671-73024ad5736d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:47:12.120343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e928-6d8f-4da4-9671-73024ad5736d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:47:12.268643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 486/983] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 12:47:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 12:47:21.976963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=69 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:47:22.264001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:54:59.530378 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1ac0f-58f8-4cff-8085-75486e048dad None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:54:59.530871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1ac0f-58f8-4cff-8085-75486e048dad None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:54:59.530993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1ac0f-58f8-4cff-8085-75486e048dad None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:54:59.532843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 498/984] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:54:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:54:59.538852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6cd41-c7c5-4581-9692-a7c2fd4c3c6f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:54:59.544809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6cd41-c7c5-4581-9692-a7c2fd4c3c6f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:54:59.545449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6cd41-c7c5-4581-9692-a7c2fd4c3c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:54:59.648921 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-80c6cd41-c7c5-4581-9692-a7c2fd4c3c6f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:54:59.650518 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80c6cd41-c7c5-4581-9692-a7c2fd4c3c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:54:59.762092 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 487/985] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:00.054887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b04f8-e1bd-4e8f-9775-b87a010b222f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:00.055424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b04f8-e1bd-4e8f-9775-b87a010b222f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:00.056563 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b04f8-e1bd-4e8f-9775-b87a010b222f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:00.058044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 499/986] 10.222.0.30 () {54 vars in 964 bytes} [Sat Oct 15 12:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:00.064638 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e0ce7-c5ac-44d7-a1eb-5c374b9792ce None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:00.064930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e0ce7-c5ac-44d7-a1eb-5c374b9792ce None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:00.065394 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e0ce7-c5ac-44d7-a1eb-5c374b9792ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:00.157452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9e3e0ce7-c5ac-44d7-a1eb-5c374b9792ce None None] MFA Rules not processed for user `7a459d7bb6ea42ed93c5bd8eea8a37b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:00.158677 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9e3e0ce7-c5ac-44d7-a1eb-5c374b9792ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:00.183829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:00.241017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 488/987] 10.222.0.30 () {58 vars in 1061 bytes} [Sat Oct 15 12:55:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:00.250945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b04f8-e1bd-4e8f-9775-b87a010b222f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:00.252741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-886b04f8-e1bd-4e8f-9775-b87a010b222f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:00.363732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b04f8-e1bd-4e8f-9775-b87a010b222f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:00.440679 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181abea-e356-4197-a159-8acfdb2f7cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:00.441231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181abea-e356-4197-a159-8acfdb2f7cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:00.441510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181abea-e356-4197-a159-8acfdb2f7cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:00.441777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181abea-e356-4197-a159-8acfdb2f7cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:00.443372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3181abea-e356-4197-a159-8acfdb2f7cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:00.501341 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:00.559702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181abea-e356-4197-a159-8acfdb2f7cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:00.561181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181abea-e356-4197-a159-8acfdb2f7cb3 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:00.583194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 500/988] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:03.085645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a873ef-f268-4dcd-b413-53aeec0a7086 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:03.086160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a873ef-f268-4dcd-b413-53aeec0a7086 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:03.086241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a873ef-f268-4dcd-b413-53aeec0a7086 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:03.091299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 489/989] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:03.094092 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a6c60-e366-4004-b7fe-69a69e82d52d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:03.094628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a6c60-e366-4004-b7fe-69a69e82d52d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:03.095067 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a6c60-e366-4004-b7fe-69a69e82d52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:03.172856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c41a6c60-e366-4004-b7fe-69a69e82d52d None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:03.174185 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c41a6c60-e366-4004-b7fe-69a69e82d52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:03.279712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 501/990] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:03.565775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a873ef-f268-4dcd-b413-53aeec0a7086 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:03.567340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-03a873ef-f268-4dcd-b413-53aeec0a7086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:03.674806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a873ef-f268-4dcd-b413-53aeec0a7086 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:03.744726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:03.746998 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cce946-8fb8-409d-a8e7-7ba6dc42cb73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:03.747603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cce946-8fb8-409d-a8e7-7ba6dc42cb73 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:03.747880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cce946-8fb8-409d-a8e7-7ba6dc42cb73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:03.748150 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cce946-8fb8-409d-a8e7-7ba6dc42cb73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:03.749757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-05cce946-8fb8-409d-a8e7-7ba6dc42cb73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:03.859282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cce946-8fb8-409d-a8e7-7ba6dc42cb73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:03.860623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cce946-8fb8-409d-a8e7-7ba6dc42cb73 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:03.878231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 490/991] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:06.682398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9223f0-b0f7-43d8-99d5-7731855be5a6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:06.682952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9223f0-b0f7-43d8-99d5-7731855be5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:06.682952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9223f0-b0f7-43d8-99d5-7731855be5a6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:06.684283 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 502/992] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:06.689448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7aba6-da05-4b25-a31a-ba71644cdf75 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:06.689864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7aba6-da05-4b25-a31a-ba71644cdf75 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:06.689999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7aba6-da05-4b25-a31a-ba71644cdf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:06.755628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4de7aba6-da05-4b25-a31a-ba71644cdf75 None None] MFA Rules not processed for user `98eb25bc1113448eb680b306b130079c`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:06.756834 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4de7aba6-da05-4b25-a31a-ba71644cdf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:06.809264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:06.852542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 491/993] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:07.136458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9223f0-b0f7-43d8-99d5-7731855be5a6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:07.138047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cc9223f0-b0f7-43d8-99d5-7731855be5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:07.147811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:07.240918 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9223f0-b0f7-43d8-99d5-7731855be5a6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:07.314192 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7fefd-5860-482b-bb41-f3d220570f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:07.314802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7fefd-5860-482b-bb41-f3d220570f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:07.315086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7fefd-5860-482b-bb41-f3d220570f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:07.315364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7fefd-5860-482b-bb41-f3d220570f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:07.316969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-edb7fefd-5860-482b-bb41-f3d220570f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:07.378726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:07.423852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7fefd-5860-482b-bb41-f3d220570f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:07.425256 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7fefd-5860-482b-bb41-f3d220570f87 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:07.443844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 503/994] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:10.332104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf0ad5-4dbb-4ff8-85f5-4d33499c9ea0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:10.332669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf0ad5-4dbb-4ff8-85f5-4d33499c9ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:10.332669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf0ad5-4dbb-4ff8-85f5-4d33499c9ea0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:10.334018 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 492/995] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:10.339280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3870a2-f1f7-4f20-828b-34cefd78d88c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:10.339715 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3870a2-f1f7-4f20-828b-34cefd78d88c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:10.339841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3870a2-f1f7-4f20-828b-34cefd78d88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:10.405693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1b3870a2-f1f7-4f20-828b-34cefd78d88c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:10.406910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1b3870a2-f1f7-4f20-828b-34cefd78d88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:10.502397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 504/996] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:10.782606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf0ad5-4dbb-4ff8-85f5-4d33499c9ea0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:10.784145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aacf0ad5-4dbb-4ff8-85f5-4d33499c9ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:10.885603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf0ad5-4dbb-4ff8-85f5-4d33499c9ea0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:10.957352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be3d72-f118-4b94-a984-476c5fae5fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:10.957956 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be3d72-f118-4b94-a984-476c5fae5fec service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:10.958229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be3d72-f118-4b94-a984-476c5fae5fec service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:10.958494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be3d72-f118-4b94-a984-476c5fae5fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:10.960084 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-49be3d72-f118-4b94-a984-476c5fae5fec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:10.967738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:11.066761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be3d72-f118-4b94-a984-476c5fae5fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:11.068186 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be3d72-f118-4b94-a984-476c5fae5fec service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:11.087902 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 493/997] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:14.247742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0234fea-66e4-4f87-ac60-9781957007c9 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:14.248308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0234fea-66e4-4f87-ac60-9781957007c9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:14.248308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0234fea-66e4-4f87-ac60-9781957007c9 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:14.249669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 505/998] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:14.255001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37018a85-aa94-4214-9199-ee5cd9bd9d40 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:14.255408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37018a85-aa94-4214-9199-ee5cd9bd9d40 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:14.255552 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37018a85-aa94-4214-9199-ee5cd9bd9d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:14.323885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-37018a85-aa94-4214-9199-ee5cd9bd9d40 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:14.325141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-37018a85-aa94-4214-9199-ee5cd9bd9d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:14.388531 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:14.442354 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 494/999] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:14.757473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4aa0b-4ee0-4e57-818e-8f5e77b360b3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:14.757866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4aa0b-4ee0-4e57-818e-8f5e77b360b3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:14.758267 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4aa0b-4ee0-4e57-818e-8f5e77b360b3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:14.760606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 506/1000] 10.222.0.30 () {54 vars in 964 bytes} [Sat Oct 15 12:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:14.766942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9517a2-fcd9-4965-aae4-e6d6eb926463 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:14.767303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9517a2-fcd9-4965-aae4-e6d6eb926463 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:14.767652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9517a2-fcd9-4965-aae4-e6d6eb926463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:14.845565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-af9517a2-fcd9-4965-aae4-e6d6eb926463 None None] MFA Rules not processed for user `7a459d7bb6ea42ed93c5bd8eea8a37b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:14.846775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-af9517a2-fcd9-4965-aae4-e6d6eb926463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:14.921651 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 495/1001] 10.222.0.30 () {58 vars in 1061 bytes} [Sat Oct 15 12:55:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:14.931482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4aa0b-4ee0-4e57-818e-8f5e77b360b3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:14.933005 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fee4aa0b-4ee0-4e57-818e-8f5e77b360b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:14.952794 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:15.040500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4aa0b-4ee0-4e57-818e-8f5e77b360b3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:15.114778 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce385d-1b9c-430f-95ed-970c1643ec53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:15.115304 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce385d-1b9c-430f-95ed-970c1643ec53 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:15.115587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce385d-1b9c-430f-95ed-970c1643ec53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:15.115861 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce385d-1b9c-430f-95ed-970c1643ec53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:15.118059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-87ce385d-1b9c-430f-95ed-970c1643ec53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:15.202095 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:15.242137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce385d-1b9c-430f-95ed-970c1643ec53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:15.244708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce385d-1b9c-430f-95ed-970c1643ec53 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:15.274869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 507/1002] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:17.705227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf166c-6732-4514-9c98-cf57e6ed054e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:17.705782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf166c-6732-4514-9c98-cf57e6ed054e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:17.705782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf166c-6732-4514-9c98-cf57e6ed054e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:17.707070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 496/1003] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:17.712697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8db75-b154-4a14-9f6d-0948256d65df None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:17.713137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8db75-b154-4a14-9f6d-0948256d65df None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:17.713308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8db75-b154-4a14-9f6d-0948256d65df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:17.779399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-13e8db75-b154-4a14-9f6d-0948256d65df None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:17.780612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-13e8db75-b154-4a14-9f6d-0948256d65df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:17.879199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 508/1004] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:18.160892 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf166c-6732-4514-9c98-cf57e6ed054e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:18.162466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dbcf166c-6732-4514-9c98-cf57e6ed054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:18.225528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:18.268471 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf166c-6732-4514-9c98-cf57e6ed054e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:18.343061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5a53f-6835-4c01-801c-6af17d5c5484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:18.343675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5a53f-6835-4c01-801c-6af17d5c5484 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:18.343955 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5a53f-6835-4c01-801c-6af17d5c5484 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:18.344231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5a53f-6835-4c01-801c-6af17d5c5484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:18.345853 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-55a5a53f-6835-4c01-801c-6af17d5c5484 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:18.454643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:18.456911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5a53f-6835-4c01-801c-6af17d5c5484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:18.458387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5a53f-6835-4c01-801c-6af17d5c5484 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:18.476394 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 497/1005] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:20.683689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b0537-70c7-4e8f-8a2a-b31ec7ea5f73 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:20.684180 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b0537-70c7-4e8f-8a2a-b31ec7ea5f73 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:20.684286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b0537-70c7-4e8f-8a2a-b31ec7ea5f73 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:20.685699 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 509/1006] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:20.691590 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e2992-03dd-4548-ad77-f6f22dbb9a68 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:20.692044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e2992-03dd-4548-ad77-f6f22dbb9a68 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:20.692265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e2992-03dd-4548-ad77-f6f22dbb9a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:20.781484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-434e2992-03dd-4548-ad77-f6f22dbb9a68 None None] MFA Rules not processed for user `98eb25bc1113448eb680b306b130079c`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:20.783043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-434e2992-03dd-4548-ad77-f6f22dbb9a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:20.893025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 498/1007] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:21.178121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b0537-70c7-4e8f-8a2a-b31ec7ea5f73 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:21.179634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f30b0537-70c7-4e8f-8a2a-b31ec7ea5f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:21.221361 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:21.295182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b0537-70c7-4e8f-8a2a-b31ec7ea5f73 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:21.368568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d40b74-45e8-46c3-ab7a-455e2fa8ec2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:21.369181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40b74-45e8-46c3-ab7a-455e2fa8ec2f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:21.369482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40b74-45e8-46c3-ab7a-455e2fa8ec2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:21.369749 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40b74-45e8-46c3-ab7a-455e2fa8ec2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:21.371848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-90d40b74-45e8-46c3-ab7a-455e2fa8ec2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:21.452612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:21.486448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d40b74-45e8-46c3-ab7a-455e2fa8ec2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:21.487856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d40b74-45e8-46c3-ab7a-455e2fa8ec2f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:21.506187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 510/1008] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:24.056231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a03ec5-6d1f-4e26-8b1e-bbb7a111133b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:24.056759 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a03ec5-6d1f-4e26-8b1e-bbb7a111133b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:24.056854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a03ec5-6d1f-4e26-8b1e-bbb7a111133b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:24.058172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 499/1009] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:24.063444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f5ac0-0aad-4c97-a5c4-2cbdea5bce8d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:24.063843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f5ac0-0aad-4c97-a5c4-2cbdea5bce8d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:24.064000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f5ac0-0aad-4c97-a5c4-2cbdea5bce8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:24.129667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8e9f5ac0-0aad-4c97-a5c4-2cbdea5bce8d None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:24.130874 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e9f5ac0-0aad-4c97-a5c4-2cbdea5bce8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:24.226347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 511/1010] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:24.507137 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a03ec5-6d1f-4e26-8b1e-bbb7a111133b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:24.508683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e4a03ec5-6d1f-4e26-8b1e-bbb7a111133b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:24.577956 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:24.611613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a03ec5-6d1f-4e26-8b1e-bbb7a111133b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:24.684706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ccf6e-de91-4517-b718-6304f5c32ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:24.685397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ccf6e-de91-4517-b718-6304f5c32ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:24.685676 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ccf6e-de91-4517-b718-6304f5c32ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:24.685950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ccf6e-de91-4517-b718-6304f5c32ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:24.687537 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-793ccf6e-de91-4517-b718-6304f5c32ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:24.795104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ccf6e-de91-4517-b718-6304f5c32ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:24.796526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ccf6e-de91-4517-b718-6304f5c32ce7 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:24.806206 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:24.814048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 500/1011] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:27.003593 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c6f71-9691-435f-9be6-29ca9fca9f5b None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:27.004111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c6f71-9691-435f-9be6-29ca9fca9f5b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:27.004204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c6f71-9691-435f-9be6-29ca9fca9f5b None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:27.005533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 512/1012] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:27.011299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0038ea0-80b8-4ede-99cb-41512bd25cd7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:27.011684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0038ea0-80b8-4ede-99cb-41512bd25cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:27.011860 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0038ea0-80b8-4ede-99cb-41512bd25cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:27.078434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f0038ea0-80b8-4ede-99cb-41512bd25cd7 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:27.079578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f0038ea0-80b8-4ede-99cb-41512bd25cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:27.178105 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 501/1013] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:27.465823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c6f71-9691-435f-9be6-29ca9fca9f5b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:27.467382 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f24c6f71-9691-435f-9be6-29ca9fca9f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:27.508509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:27.572357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c6f71-9691-435f-9be6-29ca9fca9f5b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:27.644398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cf4c5-7699-4eb6-a677-cb9d36976e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:27.645007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cf4c5-7699-4eb6-a677-cb9d36976e17 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:27.645321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cf4c5-7699-4eb6-a677-cb9d36976e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:27.645597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cf4c5-7699-4eb6-a677-cb9d36976e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:27.647180 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6f4cf4c5-7699-4eb6-a677-cb9d36976e17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:27.728213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:27.752616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cf4c5-7699-4eb6-a677-cb9d36976e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:27.754088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cf4c5-7699-4eb6-a677-cb9d36976e17 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:27.772290 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 513/1014] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:30.576472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f12cf-3bc2-42a5-b7e4-048ce45b5d70 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:30.576894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f12cf-3bc2-42a5-b7e4-048ce45b5d70 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:30.577025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f12cf-3bc2-42a5-b7e4-048ce45b5d70 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:30.578393 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 502/1015] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:30.583572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48da2e-f884-43e1-9df1-34b1ab3cf95a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:30.583965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48da2e-f884-43e1-9df1-34b1ab3cf95a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:30.584116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48da2e-f884-43e1-9df1-34b1ab3cf95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:30.648020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fb48da2e-f884-43e1-9df1-34b1ab3cf95a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:30.649294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fb48da2e-f884-43e1-9df1-34b1ab3cf95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:30.746054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 514/1016] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:31.026198 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f12cf-3bc2-42a5-b7e4-048ce45b5d70 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:31.027744 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1c4f12cf-3bc2-42a5-b7e4-048ce45b5d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:31.105367 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:31.132559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f12cf-3bc2-42a5-b7e4-048ce45b5d70 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:31.211396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5444c-8d7d-452e-9ef6-d64aeb6b421c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:31.212083 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5444c-8d7d-452e-9ef6-d64aeb6b421c service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:31.212363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5444c-8d7d-452e-9ef6-d64aeb6b421c service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:31.212703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5444c-8d7d-452e-9ef6-d64aeb6b421c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:31.214270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9db5444c-8d7d-452e-9ef6-d64aeb6b421c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:31.323560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5444c-8d7d-452e-9ef6-d64aeb6b421c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:31.324983 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5444c-8d7d-452e-9ef6-d64aeb6b421c service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:31.343873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 503/1017] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:35.623353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c078ecd-a457-46f5-855e-da29a0b2d8fa None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:35.623898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c078ecd-a457-46f5-855e-da29a0b2d8fa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:35.623898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c078ecd-a457-46f5-855e-da29a0b2d8fa None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:35.625259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 515/1018] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:35.630412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef39f0-8857-4ee0-a13d-3e07170d7ef9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:35.630800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef39f0-8857-4ee0-a13d-3e07170d7ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:35.630952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef39f0-8857-4ee0-a13d-3e07170d7ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:35.636661 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:35.699011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-83ef39f0-8857-4ee0-a13d-3e07170d7ef9 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:35.700221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-83ef39f0-8857-4ee0-a13d-3e07170d7ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:35.796369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 504/1019] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:36.083570 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c078ecd-a457-46f5-855e-da29a0b2d8fa None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:36.085374 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0c078ecd-a457-46f5-855e-da29a0b2d8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:36.135190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:36.190502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c078ecd-a457-46f5-855e-da29a0b2d8fa None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:36.265091 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c24252-e01a-498d-a373-9b7d054180e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:36.265728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c24252-e01a-498d-a373-9b7d054180e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:36.266011 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c24252-e01a-498d-a373-9b7d054180e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:36.266284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c24252-e01a-498d-a373-9b7d054180e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:36.267879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-90c24252-e01a-498d-a373-9b7d054180e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:36.370924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:36.387272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c24252-e01a-498d-a373-9b7d054180e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:36.388810 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c24252-e01a-498d-a373-9b7d054180e4 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:36.407481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 516/1020] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:39.662623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3cb0-57cd-48b8-8e94-5312c6dac6cc None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:39.663148 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3cb0-57cd-48b8-8e94-5312c6dac6cc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:39.663245 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3cb0-57cd-48b8-8e94-5312c6dac6cc None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:39.665419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 505/1021] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:39.670877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e010396-5ee8-4e32-bec0-9168337a7605 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:39.671298 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e010396-5ee8-4e32-bec0-9168337a7605 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:39.671427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e010396-5ee8-4e32-bec0-9168337a7605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:39.737813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7e010396-5ee8-4e32-bec0-9168337a7605 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:39.739068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7e010396-5ee8-4e32-bec0-9168337a7605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:39.839641 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 517/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:40.124406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf3cb0-57cd-48b8-8e94-5312c6dac6cc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:40.125994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2caf3cb0-57cd-48b8-8e94-5312c6dac6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:40.216073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:40.232166 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf3cb0-57cd-48b8-8e94-5312c6dac6cc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:40.328233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beed79b-6c07-4b44-bf41-b319e4280a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:40.328858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beed79b-6c07-4b44-bf41-b319e4280a4b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:40.329137 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beed79b-6c07-4b44-bf41-b319e4280a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:40.329450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beed79b-6c07-4b44-bf41-b319e4280a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:40.331079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1beed79b-6c07-4b44-bf41-b319e4280a4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:40.450051 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beed79b-6c07-4b44-bf41-b319e4280a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:40.451531 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beed79b-6c07-4b44-bf41-b319e4280a4b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:40.470920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 506/1023] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:45.392688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a92ab-2c9c-4ebe-a631-d6bd06d26190 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:45.393164 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a92ab-2c9c-4ebe-a631-d6bd06d26190 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:45.393332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a92ab-2c9c-4ebe-a631-d6bd06d26190 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:45.394615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 518/1024] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:45.400543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdd31b-7d16-4dc3-9d7a-da5ccb0907fe None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:45.401038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdd31b-7d16-4dc3-9d7a-da5ccb0907fe None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:45.402279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdd31b-7d16-4dc3-9d7a-da5ccb0907fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:45.418299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:45.471142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-31bdd31b-7d16-4dc3-9d7a-da5ccb0907fe None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:45.472353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-31bdd31b-7d16-4dc3-9d7a-da5ccb0907fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:45.578048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 507/1025] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:45.863277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a92ab-2c9c-4ebe-a631-d6bd06d26190 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:55:45.864822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-122a92ab-2c9c-4ebe-a631-d6bd06d26190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:45.921440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:45.967700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a92ab-2c9c-4ebe-a631-d6bd06d26190 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:55:46.039234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e92b3e-8ce3-4e82-a5bb-15340326278f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:55:46.039834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e92b3e-8ce3-4e82-a5bb-15340326278f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:46.040115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e92b3e-8ce3-4e82-a5bb-15340326278f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:46.040386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e92b3e-8ce3-4e82-a5bb-15340326278f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:46.042003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f9e92b3e-8ce3-4e82-a5bb-15340326278f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:46.145708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:46.149670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e92b3e-8ce3-4e82-a5bb-15340326278f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:55:46.151111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e92b3e-8ce3-4e82-a5bb-15340326278f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:55:46.169513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 519/1026] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:55:51.182620 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc5d7-7b23-4ded-a4c0-dc10e0557c29 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:51.183043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc5d7-7b23-4ded-a4c0-dc10e0557c29 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:51.183239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc5d7-7b23-4ded-a4c0-dc10e0557c29 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:51.184810 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 508/1027] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 12:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:51.189810 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0248b-5eac-453b-bc94-a34de071cc7f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:51.190230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0248b-5eac-453b-bc94-a34de071cc7f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:51.190400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0248b-5eac-453b-bc94-a34de071cc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:51.304994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1be0248b-5eac-453b-bc94-a34de071cc7f None None] MFA Rules not processed for user `18df1f401bb2494299eda9582c85a04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:51.306606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1be0248b-5eac-453b-bc94-a34de071cc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:51.499233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 520/1028] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 12:55:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:55.575747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=59 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:55.664668 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919a720-1ec7-4bb4-85a4-be9c58dc1e15 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:55.665045 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919a720-1ec7-4bb4-85a4-be9c58dc1e15 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:55.665299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919a720-1ec7-4bb4-85a4-be9c58dc1e15 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:55.666756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 509/1029] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 12:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:55.671923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563ccfd-0797-4789-b59c-c16970a92f69 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:55.672377 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563ccfd-0797-4789-b59c-c16970a92f69 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:55.672473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563ccfd-0797-4789-b59c-c16970a92f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:55.728814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:55.752183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2563ccfd-0797-4789-b59c-c16970a92f69 None None] MFA Rules not processed for user `18df1f401bb2494299eda9582c85a04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:55.753483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2563ccfd-0797-4789-b59c-c16970a92f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:55.864472 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 521/1030] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 12:55:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:55:57.441738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb6d7d-4562-4155-a878-77a5e9781a0a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:57.442264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb6d7d-4562-4155-a878-77a5e9781a0a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:57.442359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb6d7d-4562-4155-a878-77a5e9781a0a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:57.443689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 510/1031] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 12:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:55:57.450755 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2018e8-59b8-495b-bfc9-3737001e7a88 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:55:57.451421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2018e8-59b8-495b-bfc9-3737001e7a88 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:55:57.451421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2018e8-59b8-495b-bfc9-3737001e7a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:55:57.528807 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0b2018e8-59b8-495b-bfc9-3737001e7a88 None None] MFA Rules not processed for user `18df1f401bb2494299eda9582c85a04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:55:57.530052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0b2018e8-59b8-495b-bfc9-3737001e7a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:55:57.573331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:55:57.646215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 522/1032] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 12:55:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 12:56:01.102055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cb318-3853-44dd-9080-e6921ddf4375 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:01.102590 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cb318-3853-44dd-9080-e6921ddf4375 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:01.102762 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cb318-3853-44dd-9080-e6921ddf4375 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:01.104206 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 511/1033] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:01.113753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43d9e8-b261-4168-b4a9-2460c4d0f36f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:01.114619 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43d9e8-b261-4168-b4a9-2460c4d0f36f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:01.115220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43d9e8-b261-4168-b4a9-2460c4d0f36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:01.196298 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ac43d9e8-b261-4168-b4a9-2460c4d0f36f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:01.197566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ac43d9e8-b261-4168-b4a9-2460c4d0f36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:01.305328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 523/1034] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:01.607980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7e6f9-6f88-4eb9-a39e-2a1629f80711 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:01.608922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7e6f9-6f88-4eb9-a39e-2a1629f80711 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:01.609567 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7e6f9-6f88-4eb9-a39e-2a1629f80711 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:01.611106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 512/1035] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 12:56:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:01.619435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb6c30-4ef3-4f8a-a6cb-99a826e00a63 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:01.619728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb6c30-4ef3-4f8a-a6cb-99a826e00a63 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:01.620799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb6c30-4ef3-4f8a-a6cb-99a826e00a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:01.637217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:01.712052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f6fb6c30-4ef3-4f8a-a6cb-99a826e00a63 None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:01.713532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f6fb6c30-4ef3-4f8a-a6cb-99a826e00a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:01.847711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 524/1036] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 12:56:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:01.856267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7e6f9-6f88-4eb9-a39e-2a1629f80711 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:01.857816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-92a7e6f9-6f88-4eb9-a39e-2a1629f80711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:02.011964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7e6f9-6f88-4eb9-a39e-2a1629f80711 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:02.091324 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:02.133031 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c143a-b5f3-49e4-9cce-f27d15880db9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:02.133797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c143a-b5f3-49e4-9cce-f27d15880db9 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:02.133993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c143a-b5f3-49e4-9cce-f27d15880db9 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:02.134274 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c143a-b5f3-49e4-9cce-f27d15880db9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:02.135867 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8a6c143a-b5f3-49e4-9cce-f27d15880db9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:02.240867 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c143a-b5f3-49e4-9cce-f27d15880db9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:02.243204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c143a-b5f3-49e4-9cce-f27d15880db9 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:02.262237 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 513/1037] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:04.648894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d0170-7413-4609-ba30-a9b6740704e3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:04.649919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d0170-7413-4609-ba30-a9b6740704e3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:04.650367 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d0170-7413-4609-ba30-a9b6740704e3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:04.656242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 525/1038] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:04.657713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d2a81-cb9a-4b88-ad92-4ef2d0fc09e5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:04.658163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d2a81-cb9a-4b88-ad92-4ef2d0fc09e5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:04.658585 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d2a81-cb9a-4b88-ad92-4ef2d0fc09e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:04.764680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6b0d2a81-cb9a-4b88-ad92-4ef2d0fc09e5 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:04.766033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6b0d2a81-cb9a-4b88-ad92-4ef2d0fc09e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:04.808653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:04.871937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 514/1039] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:05.161400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d0170-7413-4609-ba30-a9b6740704e3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:05.163509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6c7d0170-7413-4609-ba30-a9b6740704e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:05.236317 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:05.336364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d0170-7413-4609-ba30-a9b6740704e3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:05.475409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb5d52-a540-4610-97ec-ffb5eb86da1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:05.475409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb5d52-a540-4610-97ec-ffb5eb86da1a service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:05.475409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb5d52-a540-4610-97ec-ffb5eb86da1a service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:05.475409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb5d52-a540-4610-97ec-ffb5eb86da1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:05.475872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e0eb5d52-a540-4610-97ec-ffb5eb86da1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:05.501505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:05.576050 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb5d52-a540-4610-97ec-ffb5eb86da1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:05.577745 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb5d52-a540-4610-97ec-ffb5eb86da1a service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:05.596839 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 526/1040] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:08.258906 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56074f3-b180-4a45-a2ff-ff1162f52f20 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:08.259440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56074f3-b180-4a45-a2ff-ff1162f52f20 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:08.259440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56074f3-b180-4a45-a2ff-ff1162f52f20 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:08.260888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 515/1041] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:56:08.265980 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619de3a-abad-48bd-9aec-90da9ac4d816 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:08.266414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619de3a-abad-48bd-9aec-90da9ac4d816 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:08.266512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619de3a-abad-48bd-9aec-90da9ac4d816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:08.337278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9619de3a-abad-48bd-9aec-90da9ac4d816 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:08.338608 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9619de3a-abad-48bd-9aec-90da9ac4d816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:08.443579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:08.453553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 527/1042] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:08.736560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8fcf5-dc44-42c1-a288-25b8c50f1faa None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:08.737643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8fcf5-dc44-42c1-a288-25b8c50f1faa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:08.738034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8fcf5-dc44-42c1-a288-25b8c50f1faa None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:08.739505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 516/1043] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 12:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:08.745591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efae2-b537-46ec-a01b-4e33fe9d4c0b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:08.745885 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efae2-b537-46ec-a01b-4e33fe9d4c0b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:08.746915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efae2-b537-46ec-a01b-4e33fe9d4c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:08.831324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-183efae2-b537-46ec-a01b-4e33fe9d4c0b None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:08.832535 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-183efae2-b537-46ec-a01b-4e33fe9d4c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:08.956712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 528/1044] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 12:56:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:08.964884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8fcf5-dc44-42c1-a288-25b8c50f1faa None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:08.966429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-21a8fcf5-dc44-42c1-a288-25b8c50f1faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:09.126419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8fcf5-dc44-42c1-a288-25b8c50f1faa None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:09.149945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:09.262821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee67ba-f8a6-43ac-9d77-7175bbf02b5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:09.263520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee67ba-f8a6-43ac-9d77-7175bbf02b5a service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:09.263709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee67ba-f8a6-43ac-9d77-7175bbf02b5a service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:09.263996 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee67ba-f8a6-43ac-9d77-7175bbf02b5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:09.265617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2dee67ba-f8a6-43ac-9d77-7175bbf02b5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:09.318588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945225d-3830-453c-9939-3345c5aec41f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:09.319727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945225d-3830-453c-9939-3345c5aec41f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:09.319727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945225d-3830-453c-9939-3345c5aec41f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:09.321248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 529/1045] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:09.326219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f991b7f-0897-4cf3-92eb-83960f722bba None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:09.326505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f991b7f-0897-4cf3-92eb-83960f722bba None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:09.326779 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f991b7f-0897-4cf3-92eb-83960f722bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:09.368503 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee67ba-f8a6-43ac-9d77-7175bbf02b5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:09.371234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee67ba-f8a6-43ac-9d77-7175bbf02b5a service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:09.390345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:09.395719 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 517/1046] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:09.398428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1f991b7f-0897-4cf3-92eb-83960f722bba None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:09.399711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1f991b7f-0897-4cf3-92eb-83960f722bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:09.508226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 530/1047] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 12:56:09.847511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d629a-6435-4a73-b5f4-4ab9b569b592 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:09.847801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d629a-6435-4a73-b5f4-4ab9b569b592 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:09.848079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d629a-6435-4a73-b5f4-4ab9b569b592 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:09.853242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 518/1048] 10.222.0.30 () {54 vars in 963 bytes} [Sat Oct 15 12:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:09.855439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13b1e9-4bdf-48cd-8064-5e9af431a499 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:09.855732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13b1e9-4bdf-48cd-8064-5e9af431a499 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:09.856011 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13b1e9-4bdf-48cd-8064-5e9af431a499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:09.936881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bb13b1e9-4bdf-48cd-8064-5e9af431a499 None None] MFA Rules not processed for user `264bc0b947614ec2a554d627e9e2da56`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:09.938087 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bb13b1e9-4bdf-48cd-8064-5e9af431a499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:10.016124 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:10.025027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 531/1049] 10.222.0.30 () {58 vars in 1060 bytes} [Sat Oct 15 12:56:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:10.032967 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d629a-6435-4a73-b5f4-4ab9b569b592 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:10.034664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8c1d629a-6435-4a73-b5f4-4ab9b569b592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:10.070911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:10.152681 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d629a-6435-4a73-b5f4-4ab9b569b592 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:10.233792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f06f2-79d2-4691-a3b0-d1ec7ce00c99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:10.234315 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f06f2-79d2-4691-a3b0-d1ec7ce00c99 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:10.234592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f06f2-79d2-4691-a3b0-d1ec7ce00c99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:10.234864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f06f2-79d2-4691-a3b0-d1ec7ce00c99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:10.236456 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4e6f06f2-79d2-4691-a3b0-d1ec7ce00c99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:10.326846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:10.360537 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f06f2-79d2-4691-a3b0-d1ec7ce00c99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:10.362010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f06f2-79d2-4691-a3b0-d1ec7ce00c99 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:10.382578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 519/1050] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:11.824401 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4880b5-4401-4b7e-9b2f-dfd66fd994b8 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:11.824903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4880b5-4401-4b7e-9b2f-dfd66fd994b8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:11.824903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4880b5-4401-4b7e-9b2f-dfd66fd994b8 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:11.826429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 532/1051] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:11.831473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929acac4-ae15-4127-8221-4c600ccaaa5a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:11.831915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929acac4-ae15-4127-8221-4c600ccaaa5a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:11.832011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929acac4-ae15-4127-8221-4c600ccaaa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:11.901382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-929acac4-ae15-4127-8221-4c600ccaaa5a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:11.902624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-929acac4-ae15-4127-8221-4c600ccaaa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:12.000464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 520/1052] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 12:56:12.288334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4880b5-4401-4b7e-9b2f-dfd66fd994b8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:12.289968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5a4880b5-4401-4b7e-9b2f-dfd66fd994b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:12.446131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4880b5-4401-4b7e-9b2f-dfd66fd994b8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:12.520708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:12.573994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ee820-e6b1-4605-948e-272f14017855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:12.574613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee820-e6b1-4605-948e-272f14017855 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:12.574925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee820-e6b1-4605-948e-272f14017855 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:12.575199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee820-e6b1-4605-948e-272f14017855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:12.576840 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-718ee820-e6b1-4605-948e-272f14017855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:12.674413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ee820-e6b1-4605-948e-272f14017855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:12.676041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ee820-e6b1-4605-948e-272f14017855 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:12.695481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 533/1053] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:15.091067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8377f-bfab-4368-98aa-78f2c8a17f34 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:15.091567 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8377f-bfab-4368-98aa-78f2c8a17f34 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:15.091567 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8377f-bfab-4368-98aa-78f2c8a17f34 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:15.092994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 521/1054] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:15.098194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71deff-5f2d-4c9f-921a-80de1d89bea9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:15.098609 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71deff-5f2d-4c9f-921a-80de1d89bea9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:15.098787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71deff-5f2d-4c9f-921a-80de1d89bea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:15.170181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-db71deff-5f2d-4c9f-921a-80de1d89bea9 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:15.171382 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db71deff-5f2d-4c9f-921a-80de1d89bea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:15.203979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:15.278593 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 534/1055] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:15.573880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8377f-bfab-4368-98aa-78f2c8a17f34 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:15.576520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b4d8377f-bfab-4368-98aa-78f2c8a17f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:15.624061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:15.750331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8377f-bfab-4368-98aa-78f2c8a17f34 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:15.893314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:15.899375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ab3ff-5a3a-4de0-a437-47309e48bdd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:15.900210 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ab3ff-5a3a-4de0-a437-47309e48bdd9 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:15.907728 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ab3ff-5a3a-4de0-a437-47309e48bdd9 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:15.907728 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ab3ff-5a3a-4de0-a437-47309e48bdd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:15.907728 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cd9ab3ff-5a3a-4de0-a437-47309e48bdd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:16.001520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ab3ff-5a3a-4de0-a437-47309e48bdd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:16.003340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ab3ff-5a3a-4de0-a437-47309e48bdd9 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:16.023995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 522/1056] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:18.441136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9e87e-983d-472f-aaa5-c20be91d02af None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:18.441703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9e87e-983d-472f-aaa5-c20be91d02af None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:18.441703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9e87e-983d-472f-aaa5-c20be91d02af None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:18.443002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 535/1057] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:18.448384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24284-486a-4200-9105-54789ad9c168 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:18.448798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24284-486a-4200-9105-54789ad9c168 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:18.448937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24284-486a-4200-9105-54789ad9c168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:18.523071 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-81a24284-486a-4200-9105-54789ad9c168 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:18.524305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-81a24284-486a-4200-9105-54789ad9c168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:18.596331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:18.630498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 523/1058] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:18.945869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9e87e-983d-472f-aaa5-c20be91d02af None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:18.947406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-33b9e87e-983d-472f-aaa5-c20be91d02af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:19.101046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9e87e-983d-472f-aaa5-c20be91d02af None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:19.114229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:19.221499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4b223-e56a-408d-83c1-7ce687fc2edc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:19.222111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4b223-e56a-408d-83c1-7ce687fc2edc service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:19.222386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4b223-e56a-408d-83c1-7ce687fc2edc service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:19.222655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4b223-e56a-408d-83c1-7ce687fc2edc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:19.224448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cdd4b223-e56a-408d-83c1-7ce687fc2edc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:19.324068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4b223-e56a-408d-83c1-7ce687fc2edc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:19.326495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4b223-e56a-408d-83c1-7ce687fc2edc service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:19.344974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 536/1059] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:21.739934 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37922821-db80-4cb5-aed9-1855da609d29 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:21.741058 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37922821-db80-4cb5-aed9-1855da609d29 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:21.741058 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37922821-db80-4cb5-aed9-1855da609d29 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:21.742436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 524/1060] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:21.748161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e61208-c291-4736-9625-c10305feb0b4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:21.748788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e61208-c291-4736-9625-c10305feb0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:21.748788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e61208-c291-4736-9625-c10305feb0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:21.771520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:21.824904 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-17e61208-c291-4736-9625-c10305feb0b4 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:21.826212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-17e61208-c291-4736-9625-c10305feb0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:21.932894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 537/1061] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:22.229217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37922821-db80-4cb5-aed9-1855da609d29 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:22.230780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-37922821-db80-4cb5-aed9-1855da609d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:22.401662 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37922821-db80-4cb5-aed9-1855da609d29 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:22.456957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:22.519266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ed44c-3639-41aa-904b-3fa5a66354b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:22.519870 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ed44c-3639-41aa-904b-3fa5a66354b6 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:22.520148 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ed44c-3639-41aa-904b-3fa5a66354b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:22.520418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ed44c-3639-41aa-904b-3fa5a66354b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:22.522824 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e13ed44c-3639-41aa-904b-3fa5a66354b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:22.645450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ed44c-3639-41aa-904b-3fa5a66354b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:22.647815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ed44c-3639-41aa-904b-3fa5a66354b6 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:22.667293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 525/1062] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:25.235255 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce02fe-b42d-4707-ac8e-897358688a6e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:25.235782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce02fe-b42d-4707-ac8e-897358688a6e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:25.235898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce02fe-b42d-4707-ac8e-897358688a6e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:25.237546 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 538/1063] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:25.244502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff1e9a-54c2-41f3-8e30-7a3b9228ef6d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:25.244942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff1e9a-54c2-41f3-8e30-7a3b9228ef6d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:25.245346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff1e9a-54c2-41f3-8e30-7a3b9228ef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:25.315624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3bff1e9a-54c2-41f3-8e30-7a3b9228ef6d None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:25.316849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3bff1e9a-54c2-41f3-8e30-7a3b9228ef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:25.326052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:25.418392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 526/1064] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:25.707719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce02fe-b42d-4707-ac8e-897358688a6e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:25.709832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-24ce02fe-b42d-4707-ac8e-897358688a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:25.801794 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:25.856845 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce02fe-b42d-4707-ac8e-897358688a6e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:25.973236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29060c95-c55e-4e56-bd0c-eaefca0ca721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:25.973968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29060c95-c55e-4e56-bd0c-eaefca0ca721 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:25.974253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29060c95-c55e-4e56-bd0c-eaefca0ca721 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:25.975036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29060c95-c55e-4e56-bd0c-eaefca0ca721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:25.976734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-29060c95-c55e-4e56-bd0c-eaefca0ca721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:26.036592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:26.067577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29060c95-c55e-4e56-bd0c-eaefca0ca721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:26.069261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29060c95-c55e-4e56-bd0c-eaefca0ca721 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:26.088324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 539/1065] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:27.411683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a57dd-56c7-4621-9a03-1f4bdfe058bf None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:27.412268 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a57dd-56c7-4621-9a03-1f4bdfe058bf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:27.412268 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a57dd-56c7-4621-9a03-1f4bdfe058bf None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:27.414472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 527/1066] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:27.420153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd0ad1-7b9c-4918-9d3b-a0d354780268 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:27.420582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd0ad1-7b9c-4918-9d3b-a0d354780268 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:27.420719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd0ad1-7b9c-4918-9d3b-a0d354780268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:27.489004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7cbd0ad1-7b9c-4918-9d3b-a0d354780268 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:27.490304 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7cbd0ad1-7b9c-4918-9d3b-a0d354780268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:27.591852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 540/1067] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:27.896395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f6a99-8f48-4714-9489-dc408bdbe038 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:27.896754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f6a99-8f48-4714-9489-dc408bdbe038 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:27.897021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f6a99-8f48-4714-9489-dc408bdbe038 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:27.898547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 528/1068] 10.222.0.30 () {54 vars in 962 bytes} [Sat Oct 15 12:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:27.904811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a662164-d7b0-416c-ab30-a65c292cd227 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:27.905105 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a662164-d7b0-416c-ab30-a65c292cd227 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:27.905404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a662164-d7b0-416c-ab30-a65c292cd227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:27.954065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:27.985111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6a662164-d7b0-416c-ab30-a65c292cd227 None None] MFA Rules not processed for user `5fc4d28cded84afba73b053d18d4aa23`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:27.986417 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6a662164-d7b0-416c-ab30-a65c292cd227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:28.063484 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 541/1069] 10.222.0.30 () {58 vars in 1059 bytes} [Sat Oct 15 12:56:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 12:56:28.075419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f6a99-8f48-4714-9489-dc408bdbe038 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:28.077081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d04f6a99-8f48-4714-9489-dc408bdbe038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:28.185395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f6a99-8f48-4714-9489-dc408bdbe038 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:28.224045 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:28.262277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701e7cc-4d1d-44f0-a371-de0a6f645ad0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc4d28cded84afba73b053d18d4aa23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:28.262898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701e7cc-4d1d-44f0-a371-de0a6f645ad0 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:28.263180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701e7cc-4d1d-44f0-a371-de0a6f645ad0 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:28.263458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701e7cc-4d1d-44f0-a371-de0a6f645ad0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:28.265068 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4701e7cc-4d1d-44f0-a371-de0a6f645ad0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:28.382266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701e7cc-4d1d-44f0-a371-de0a6f645ad0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:28.383738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701e7cc-4d1d-44f0-a371-de0a6f645ad0 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:28.405233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 529/1070] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:28.430137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7da3b-3009-4a0f-91d1-dcaea446e12d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:28.430703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7da3b-3009-4a0f-91d1-dcaea446e12d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:28.430703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7da3b-3009-4a0f-91d1-dcaea446e12d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:28.432111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 542/1071] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:28.437529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab38a9-46d5-457a-b3eb-60ed83250e39 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:28.437982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab38a9-46d5-457a-b3eb-60ed83250e39 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:28.438090 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab38a9-46d5-457a-b3eb-60ed83250e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:28.506392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-95ab38a9-46d5-457a-b3eb-60ed83250e39 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:28.507620 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-95ab38a9-46d5-457a-b3eb-60ed83250e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:28.526319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:28.612849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 530/1072] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:28.905041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7da3b-3009-4a0f-91d1-dcaea446e12d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:28.906622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-02a7da3b-3009-4a0f-91d1-dcaea446e12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:29.036762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:29.056844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7da3b-3009-4a0f-91d1-dcaea446e12d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:29.183876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d93b1-80ed-45ba-b944-ff5df8f829a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:29.189416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d93b1-80ed-45ba-b944-ff5df8f829a6 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:29.189416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d93b1-80ed-45ba-b944-ff5df8f829a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:29.189416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d93b1-80ed-45ba-b944-ff5df8f829a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:29.189416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-117d93b1-80ed-45ba-b944-ff5df8f829a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:29.278526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d93b1-80ed-45ba-b944-ff5df8f829a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:29.280184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d93b1-80ed-45ba-b944-ff5df8f829a6 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:29.287798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:29.306607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 543/1073] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:29.309697 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b748644-d005-438a-9490-59af078fb0c7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:29.309985 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b748644-d005-438a-9490-59af078fb0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:29.310285 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b748644-d005-438a-9490-59af078fb0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:29.377029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2b748644-d005-438a-9490-59af078fb0c7 None None] MFA Rules not processed for user `16c94590fafa48f58fd94526a913dd66`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:29.378258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2b748644-d005-438a-9490-59af078fb0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:29.467146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 531/1074] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 12:56:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:29.515990 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52727a3d-d5da-430c-89d2-3a6c312070f0 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:29.516561 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52727a3d-d5da-430c-89d2-3a6c312070f0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:29.516654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52727a3d-d5da-430c-89d2-3a6c312070f0 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:29.518463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 544/1075] 10.222.0.30 () {54 vars in 943 bytes} [Sat Oct 15 12:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:29.526579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a6dfb-5413-4029-9f15-c5221b74352c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:29.526972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a6dfb-5413-4029-9f15-c5221b74352c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:29.527339 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a6dfb-5413-4029-9f15-c5221b74352c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:29.544185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:29.620625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-263a6dfb-5413-4029-9f15-c5221b74352c None None] MFA Rules not processed for user `b849f213606a4686bfb140fec34d512f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:29.621974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-263a6dfb-5413-4029-9f15-c5221b74352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:29.737563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 532/1076] 10.222.0.30 () {58 vars in 1040 bytes} [Sat Oct 15 12:56:29 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 12:56:29.747520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52727a3d-d5da-430c-89d2-3a6c312070f0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:29.749151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-52727a3d-d5da-430c-89d2-3a6c312070f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:29.884998 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52727a3d-d5da-430c-89d2-3a6c312070f0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:29.957509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:29.987819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2387be-49a9-42e5-8df2-2c4c29315f14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b849f213606a4686bfb140fec34d512f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:29.988381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2387be-49a9-42e5-8df2-2c4c29315f14 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:29.988662 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2387be-49a9-42e5-8df2-2c4c29315f14 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:29.988934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2387be-49a9-42e5-8df2-2c4c29315f14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:29.990829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e2387be-49a9-42e5-8df2-2c4c29315f14 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:30.075354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2387be-49a9-42e5-8df2-2c4c29315f14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:30.076964 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2387be-49a9-42e5-8df2-2c4c29315f14 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:30.085095 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 545/1077] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 12:56:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 12:56:31.687466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0041e-54a1-43fb-b12e-ef6c2636cdfd None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:31.687466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0041e-54a1-43fb-b12e-ef6c2636cdfd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:31.687466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0041e-54a1-43fb-b12e-ef6c2636cdfd None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:31.687466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 533/1078] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 12:56:31.690824 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db27fa-8335-4562-a4e4-e318c4de48a1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:31.691226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db27fa-8335-4562-a4e4-e318c4de48a1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:31.691368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db27fa-8335-4562-a4e4-e318c4de48a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:31.767049 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d9db27fa-8335-4562-a4e4-e318c4de48a1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:31.768647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d9db27fa-8335-4562-a4e4-e318c4de48a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:31.834417 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:31.878194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 546/1079] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:32.173593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0041e-54a1-43fb-b12e-ef6c2636cdfd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:32.175134 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-93e0041e-54a1-43fb-b12e-ef6c2636cdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:32.261362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:32.357660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0041e-54a1-43fb-b12e-ef6c2636cdfd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:32.495024 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da296bf7-f211-4139-8c88-4e42bcab7d9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:32.495563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da296bf7-f211-4139-8c88-4e42bcab7d9f service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:32.495844 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da296bf7-f211-4139-8c88-4e42bcab7d9f service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:32.496117 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da296bf7-f211-4139-8c88-4e42bcab7d9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:32.498182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-da296bf7-f211-4139-8c88-4e42bcab7d9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:32.604922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da296bf7-f211-4139-8c88-4e42bcab7d9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:32.606588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da296bf7-f211-4139-8c88-4e42bcab7d9f service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:32.626706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 534/1080] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:32.938111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:35.887885 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f3d65-6c2f-43db-af38-eadea840df70 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:35.888419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f3d65-6c2f-43db-af38-eadea840df70 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:35.888419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f3d65-6c2f-43db-af38-eadea840df70 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:35.889843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 547/1081] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:35.897995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3343d5-767f-4b0d-acf3-802ce30f2f9f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:35.897995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3343d5-767f-4b0d-acf3-802ce30f2f9f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:35.897995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3343d5-767f-4b0d-acf3-802ce30f2f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:35.969243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0d3343d5-767f-4b0d-acf3-802ce30f2f9f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:35.972254 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0d3343d5-767f-4b0d-acf3-802ce30f2f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:36.080208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 535/1082] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:36.373010 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f3d65-6c2f-43db-af38-eadea840df70 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:36.374572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-545f3d65-6c2f-43db-af38-eadea840df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:36.542898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f3d65-6c2f-43db-af38-eadea840df70 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:36.603792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:36.674161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1b96b-8699-4671-a88e-2e3fb9cf5435 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:36.674695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1b96b-8699-4671-a88e-2e3fb9cf5435 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:36.674978 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1b96b-8699-4671-a88e-2e3fb9cf5435 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:36.675252 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1b96b-8699-4671-a88e-2e3fb9cf5435 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:36.676883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3fa1b96b-8699-4671-a88e-2e3fb9cf5435 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:36.776044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1b96b-8699-4671-a88e-2e3fb9cf5435 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:36.777691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1b96b-8699-4671-a88e-2e3fb9cf5435 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:36.797715 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 548/1083] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:39.151645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19256-211e-4045-8eda-44234d67a1cc None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:39.152203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19256-211e-4045-8eda-44234d67a1cc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:39.152203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19256-211e-4045-8eda-44234d67a1cc None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:39.157430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 536/1084] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:56:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:39.162266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d0fff-5a1e-456c-8087-d8af2921e761 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:39.162266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d0fff-5a1e-456c-8087-d8af2921e761 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:39.162266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d0fff-5a1e-456c-8087-d8af2921e761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:39.234903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dc6d0fff-5a1e-456c-8087-d8af2921e761 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:39.236131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dc6d0fff-5a1e-456c-8087-d8af2921e761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:39.238451 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:39.344438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 549/1085] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:39.647994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19256-211e-4045-8eda-44234d67a1cc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:39.649569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-76f19256-211e-4045-8eda-44234d67a1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:39.660813 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:39.813303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19256-211e-4045-8eda-44234d67a1cc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:39.906298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:39.941990 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb64b7c-716f-4457-9de7-4eb65b721dae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:39.942595 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb64b7c-716f-4457-9de7-4eb65b721dae service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:39.942954 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb64b7c-716f-4457-9de7-4eb65b721dae service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:39.943860 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb64b7c-716f-4457-9de7-4eb65b721dae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:39.947222 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8eb64b7c-716f-4457-9de7-4eb65b721dae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:40.042838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb64b7c-716f-4457-9de7-4eb65b721dae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:40.044880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb64b7c-716f-4457-9de7-4eb65b721dae service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:40.064387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 537/1086] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:49.377910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=42 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:50.061913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=58 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:54.298723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aea90-c9d3-49cb-97f2-dcc21223ce7c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:54.299295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aea90-c9d3-49cb-97f2-dcc21223ce7c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:54.299295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aea90-c9d3-49cb-97f2-dcc21223ce7c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:54.300661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 550/1087] 10.222.0.30 () {54 vars in 918 bytes} [Sat Oct 15 12:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:54.306822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956cce5-321b-4d6d-a25b-b9ca5130fb11 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:54.307260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956cce5-321b-4d6d-a25b-b9ca5130fb11 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:54.307402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956cce5-321b-4d6d-a25b-b9ca5130fb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:54.382839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6956cce5-321b-4d6d-a25b-b9ca5130fb11 None None] MFA Rules not processed for user `18df1f401bb2494299eda9582c85a04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:54.384041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6956cce5-321b-4d6d-a25b-b9ca5130fb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:54.486735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 538/1088] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Oct 15 12:56:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:54.500951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6936e59-bd96-4d25-99c3-ade87b1a55aa None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:54.501262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6936e59-bd96-4d25-99c3-ade87b1a55aa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:54.501537 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6936e59-bd96-4d25-99c3-ade87b1a55aa None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:54.502940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 551/1089] 10.222.0.30 () {54 vars in 918 bytes} [Sat Oct 15 12:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:54.508585 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46890d05-730c-455b-85ea-b37539429655 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:54.508877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46890d05-730c-455b-85ea-b37539429655 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:54.509155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46890d05-730c-455b-85ea-b37539429655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:54.584338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:54.587465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-46890d05-730c-455b-85ea-b37539429655 None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:54.588688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-46890d05-730c-455b-85ea-b37539429655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:54.706356 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 539/1090] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Oct 15 12:56:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:57.280183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151062a8-8b79-4457-91d1-fcd77985bf8e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:57.280729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151062a8-8b79-4457-91d1-fcd77985bf8e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:57.280729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151062a8-8b79-4457-91d1-fcd77985bf8e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:57.282140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 552/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:57.287372 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796c0f1-66a8-4c2c-b43d-cfcd74157eb2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:57.287783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796c0f1-66a8-4c2c-b43d-cfcd74157eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:57.287920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796c0f1-66a8-4c2c-b43d-cfcd74157eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:57.353419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c796c0f1-66a8-4c2c-b43d-cfcd74157eb2 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:57.354625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c796c0f1-66a8-4c2c-b43d-cfcd74157eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:57.408421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:57.450373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 540/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:57.743750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151062a8-8b79-4457-91d1-fcd77985bf8e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:57.745362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-151062a8-8b79-4457-91d1-fcd77985bf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:57.895905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151062a8-8b79-4457-91d1-fcd77985bf8e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:57.969265 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:58.009326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd4bd1-ad97-4905-a522-bc1ffeadf3ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:58.010191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd4bd1-ad97-4905-a522-bc1ffeadf3ce service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:58.010664 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd4bd1-ad97-4905-a522-bc1ffeadf3ce service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:58.011264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd4bd1-ad97-4905-a522-bc1ffeadf3ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:58.012851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-56dd4bd1-ad97-4905-a522-bc1ffeadf3ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:58.102362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd4bd1-ad97-4905-a522-bc1ffeadf3ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:58.103966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd4bd1-ad97-4905-a522-bc1ffeadf3ce service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:58.122037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 553/1093] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:56:59.256561 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a2f85-24a0-45fc-bae1-bf903424a588 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:59.257070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a2f85-24a0-45fc-bae1-bf903424a588 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:59.257070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a2f85-24a0-45fc-bae1-bf903424a588 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:59.258437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 541/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 12:56:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:56:59.264778 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c7c22-27e9-43b5-911e-21a92bf405aa None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:59.265276 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c7c22-27e9-43b5-911e-21a92bf405aa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:59.265385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c7c22-27e9-43b5-911e-21a92bf405aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:59.342592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-aa6c7c22-27e9-43b5-911e-21a92bf405aa None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:56:59.343820 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-aa6c7c22-27e9-43b5-911e-21a92bf405aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:59.370799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:59.439533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 554/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 12:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:56:59.595002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a2f85-24a0-45fc-bae1-bf903424a588 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:56:59.596525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c86a2f85-24a0-45fc-bae1-bf903424a588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:59.744940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a2f85-24a0-45fc-bae1-bf903424a588 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:56:59.786493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:56:59.858695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc107b0a-5525-4c11-9842-9f0f8741e578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:56:59.859225 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc107b0a-5525-4c11-9842-9f0f8741e578 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:56:59.859505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc107b0a-5525-4c11-9842-9f0f8741e578 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:56:59.859786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc107b0a-5525-4c11-9842-9f0f8741e578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:56:59.861453 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bc107b0a-5525-4c11-9842-9f0f8741e578 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:56:59.950068 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc107b0a-5525-4c11-9842-9f0f8741e578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:56:59.951665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc107b0a-5525-4c11-9842-9f0f8741e578 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:56:59.973920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 542/1096] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:01.096168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a40b04-3ebd-402b-972a-dbec0a834a95 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:01.099262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a40b04-3ebd-402b-972a-dbec0a834a95 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:01.099262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a40b04-3ebd-402b-972a-dbec0a834a95 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:01.099262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 555/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 12:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:01.105303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2262277-cb24-4352-81a5-caa2a91b2669 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:01.105731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2262277-cb24-4352-81a5-caa2a91b2669 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:01.105940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2262277-cb24-4352-81a5-caa2a91b2669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:01.160522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:01.183748 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f2262277-cb24-4352-81a5-caa2a91b2669 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:01.185338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f2262277-cb24-4352-81a5-caa2a91b2669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:01.282231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 543/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 12:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:01.439485 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a40b04-3ebd-402b-972a-dbec0a834a95 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:01.440455 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-96a40b04-3ebd-402b-972a-dbec0a834a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:01.588233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a40b04-3ebd-402b-972a-dbec0a834a95 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:01.601139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:01.702120 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94c93-f770-4392-baed-de9d43751c2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:01.702653 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94c93-f770-4392-baed-de9d43751c2c service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:01.702932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94c93-f770-4392-baed-de9d43751c2c service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:01.703205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94c93-f770-4392-baed-de9d43751c2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:01.704801 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-69a94c93-f770-4392-baed-de9d43751c2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:01.793009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a94c93-f770-4392-baed-de9d43751c2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:01.794898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a94c93-f770-4392-baed-de9d43751c2c service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:01.813017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 556/1099] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:02.922361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf8d57-a7c9-4063-834f-c122af6954d4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:02.922889 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf8d57-a7c9-4063-834f-c122af6954d4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:02.923003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf8d57-a7c9-4063-834f-c122af6954d4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:02.924392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 544/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 12:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:02.930764 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095139f-787b-4527-9efb-2b50eb22b919 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:02.931206 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095139f-787b-4527-9efb-2b50eb22b919 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:02.931348 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095139f-787b-4527-9efb-2b50eb22b919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:02.951317 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:03.008215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a095139f-787b-4527-9efb-2b50eb22b919 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:03.009455 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a095139f-787b-4527-9efb-2b50eb22b919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:03.105505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 557/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 12:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:03.257284 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf8d57-a7c9-4063-834f-c122af6954d4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:03.258800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4cdf8d57-a7c9-4063-834f-c122af6954d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:03.368690 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:03.404597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf8d57-a7c9-4063-834f-c122af6954d4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:03.517554 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e2760-281f-4375-b9f9-dfeffb205e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:03.518157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e2760-281f-4375-b9f9-dfeffb205e99 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:03.518432 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e2760-281f-4375-b9f9-dfeffb205e99 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:03.518701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e2760-281f-4375-b9f9-dfeffb205e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:03.520381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c90e2760-281f-4375-b9f9-dfeffb205e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:03.597117 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:03.611163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e2760-281f-4375-b9f9-dfeffb205e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:03.612781 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e2760-281f-4375-b9f9-dfeffb205e99 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:03.631267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 545/1102] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:04.751892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9ec9-8e15-42b0-9210-eef6b86fbcb0 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:04.752427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9ec9-8e15-42b0-9210-eef6b86fbcb0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:04.752557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9ec9-8e15-42b0-9210-eef6b86fbcb0 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:04.753948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 558/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 12:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:04.760733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5858f4a-7e11-4fd8-8da2-51c438797176 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:04.761172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5858f4a-7e11-4fd8-8da2-51c438797176 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:04.761328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5858f4a-7e11-4fd8-8da2-51c438797176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:04.836838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c5858f4a-7e11-4fd8-8da2-51c438797176 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:04.837980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c5858f4a-7e11-4fd8-8da2-51c438797176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:04.933960 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 546/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 12:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 12:57:05.083639 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9ec9-8e15-42b0-9210-eef6b86fbcb0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:05.085163 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ad0d9ec9-8e15-42b0-9210-eef6b86fbcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:05.167939 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:05.228524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9ec9-8e15-42b0-9210-eef6b86fbcb0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:05.339359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346c736-145a-43f4-a398-24976974555a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:05.339963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346c736-145a-43f4-a398-24976974555a service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:05.340244 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346c736-145a-43f4-a398-24976974555a service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:05.340517 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346c736-145a-43f4-a398-24976974555a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:05.342153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5346c736-145a-43f4-a398-24976974555a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:05.389977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:05.431227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346c736-145a-43f4-a398-24976974555a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:05.432792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346c736-145a-43f4-a398-24976974555a service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:05.450491 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 559/1105] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:06.571361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9b867-0edc-47d7-afed-0126212262db None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:06.571841 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9b867-0edc-47d7-afed-0126212262db None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:06.571962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9b867-0edc-47d7-afed-0126212262db None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:06.573275 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 547/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 12:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:06.579669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a23a3-a710-4872-98ea-c4c037b09d21 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:06.580101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a23a3-a710-4872-98ea-c4c037b09d21 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:06.580229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a23a3-a710-4872-98ea-c4c037b09d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:06.656862 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-390a23a3-a710-4872-98ea-c4c037b09d21 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:06.658099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-390a23a3-a710-4872-98ea-c4c037b09d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:06.753657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 560/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 12:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:06.904609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9b867-0edc-47d7-afed-0126212262db None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:06.906177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cec9b867-0edc-47d7-afed-0126212262db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:06.955646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:07.052075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9b867-0edc-47d7-afed-0126212262db None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:07.167054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fcb0d-551e-4ced-9c9b-a8483b33b5ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:07.167709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcb0d-551e-4ced-9c9b-a8483b33b5ac service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:07.167988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcb0d-551e-4ced-9c9b-a8483b33b5ac service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:07.168258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcb0d-551e-4ced-9c9b-a8483b33b5ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:07.170136 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-195fcb0d-551e-4ced-9c9b-a8483b33b5ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:07.177597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:07.261118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fcb0d-551e-4ced-9c9b-a8483b33b5ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:07.262755 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fcb0d-551e-4ced-9c9b-a8483b33b5ac service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:07.281210 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 548/1108] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:08.403980 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd83c5a-2596-4cb9-9802-84628dbedfae None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:08.404553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd83c5a-2596-4cb9-9802-84628dbedfae None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:08.404553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd83c5a-2596-4cb9-9802-84628dbedfae None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:08.405921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 561/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 12:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:08.412543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74ffaa-6a8f-4ff9-beb4-fdedebd208a7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:08.412952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74ffaa-6a8f-4ff9-beb4-fdedebd208a7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:08.413097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74ffaa-6a8f-4ff9-beb4-fdedebd208a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:08.488024 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cc74ffaa-6a8f-4ff9-beb4-fdedebd208a7 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:08.489546 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cc74ffaa-6a8f-4ff9-beb4-fdedebd208a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:08.558287 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:08.586869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 549/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 12:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:08.739788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd83c5a-2596-4cb9-9802-84628dbedfae None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:08.741277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1fd83c5a-2596-4cb9-9802-84628dbedfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:08.758847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:08.887167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd83c5a-2596-4cb9-9802-84628dbedfae None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:08.972504 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:08.997689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3baf9c-840d-4669-87a1-a146eec91659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:08.998232 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3baf9c-840d-4669-87a1-a146eec91659 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:08.998514 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3baf9c-840d-4669-87a1-a146eec91659 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:08.998792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3baf9c-840d-4669-87a1-a146eec91659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:09.000402 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2f3baf9c-840d-4669-87a1-a146eec91659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:09.092340 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3baf9c-840d-4669-87a1-a146eec91659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:09.093981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3baf9c-840d-4669-87a1-a146eec91659 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:09.111681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 562/1111] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 12:57:10.251896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403160a-f7d6-45ed-937a-207c6ebd0c64 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:10.252507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403160a-f7d6-45ed-937a-207c6ebd0c64 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:10.252507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403160a-f7d6-45ed-937a-207c6ebd0c64 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:10.253835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 550/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 12:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:10.260582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d649c7-3e69-4061-9c6a-b778930762ea None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:10.261040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d649c7-3e69-4061-9c6a-b778930762ea None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:10.261155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d649c7-3e69-4061-9c6a-b778930762ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:10.335529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b4d649c7-3e69-4061-9c6a-b778930762ea None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:10.336733 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b4d649c7-3e69-4061-9c6a-b778930762ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:10.377058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:10.431502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 563/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 12:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:10.583232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403160a-f7d6-45ed-937a-207c6ebd0c64 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:10.584900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9403160a-f7d6-45ed-937a-207c6ebd0c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:10.730474 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403160a-f7d6-45ed-937a-207c6ebd0c64 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:10.785414 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:10.844214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161f8b0-de16-466d-8518-cd8d3955d831 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:10.844747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161f8b0-de16-466d-8518-cd8d3955d831 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:10.845031 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161f8b0-de16-466d-8518-cd8d3955d831 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:10.845339 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161f8b0-de16-466d-8518-cd8d3955d831 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:10.846957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b161f8b0-de16-466d-8518-cd8d3955d831 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:10.936463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161f8b0-de16-466d-8518-cd8d3955d831 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:10.938418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161f8b0-de16-466d-8518-cd8d3955d831 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:10.956120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 551/1114] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:12.062679 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3042e-ff06-4287-9e4c-221d85a3175b None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:12.063227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3042e-ff06-4287-9e4c-221d85a3175b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:12.063227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3042e-ff06-4287-9e4c-221d85a3175b None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:12.064585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 564/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 12:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:12.071632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951438a8-d032-4a17-b235-56df556f6aeb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:12.072087 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951438a8-d032-4a17-b235-56df556f6aeb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:12.072192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951438a8-d032-4a17-b235-56df556f6aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:12.147240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-951438a8-d032-4a17-b235-56df556f6aeb None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:12.148440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-951438a8-d032-4a17-b235-56df556f6aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:12.150458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:12.244075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 552/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 12:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:12.395263 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3042e-ff06-4287-9e4c-221d85a3175b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:12.396762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-16b3042e-ff06-4287-9e4c-221d85a3175b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:12.540114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3042e-ff06-4287-9e4c-221d85a3175b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:12.568059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:12.654020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94aba9b-fc02-455f-a7b0-aeeeafa2302c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:12.654578 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94aba9b-fc02-455f-a7b0-aeeeafa2302c service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:12.654853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94aba9b-fc02-455f-a7b0-aeeeafa2302c service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:12.655121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94aba9b-fc02-455f-a7b0-aeeeafa2302c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:12.656703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d94aba9b-fc02-455f-a7b0-aeeeafa2302c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:12.744665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94aba9b-fc02-455f-a7b0-aeeeafa2302c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:12.746278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94aba9b-fc02-455f-a7b0-aeeeafa2302c service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:12.763672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 565/1117] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:14.815486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee3d82-8845-4dce-a72f-5c499bc4512f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:14.816050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee3d82-8845-4dce-a72f-5c499bc4512f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:14.816050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee3d82-8845-4dce-a72f-5c499bc4512f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:14.817508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 553/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:14.822844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159877d-6fc8-41d0-bd21-7487e44eb743 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:14.823251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159877d-6fc8-41d0-bd21-7487e44eb743 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:14.823394 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159877d-6fc8-41d0-bd21-7487e44eb743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:14.864920 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:14.889162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3159877d-6fc8-41d0-bd21-7487e44eb743 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:14.890436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3159877d-6fc8-41d0-bd21-7487e44eb743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:14.985119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 566/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:15.270963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee3d82-8845-4dce-a72f-5c499bc4512f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:15.272436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9dee3d82-8845-4dce-a72f-5c499bc4512f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:15.377343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee3d82-8845-4dce-a72f-5c499bc4512f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:15.409491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:15.450724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad0c6b-6def-487a-899e-4d3a662e1421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:15.451258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad0c6b-6def-487a-899e-4d3a662e1421 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:15.451534 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad0c6b-6def-487a-899e-4d3a662e1421 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:15.451802 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad0c6b-6def-487a-899e-4d3a662e1421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:15.453414 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-51ad0c6b-6def-487a-899e-4d3a662e1421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:15.562797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad0c6b-6def-487a-899e-4d3a662e1421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:15.564283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad0c6b-6def-487a-899e-4d3a662e1421 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:15.583483 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 554/1120] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:17.922137 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055ece1-bdfb-4703-b9f8-7770e56ab2ce None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:17.922661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055ece1-bdfb-4703-b9f8-7770e56ab2ce None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:17.922661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055ece1-bdfb-4703-b9f8-7770e56ab2ce None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:17.924067 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 567/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:17.930044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0d6a3-08a4-4172-a18b-ef30d6aa5c06 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:17.930489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0d6a3-08a4-4172-a18b-ef30d6aa5c06 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:17.930594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0d6a3-08a4-4172-a18b-ef30d6aa5c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:17.998018 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6bf0d6a3-08a4-4172-a18b-ef30d6aa5c06 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:17.999233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6bf0d6a3-08a4-4172-a18b-ef30d6aa5c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:18.011902 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:18.096034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 555/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:18.379959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055ece1-bdfb-4703-b9f8-7770e56ab2ce None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:18.381626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4055ece1-bdfb-4703-b9f8-7770e56ab2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:18.482996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055ece1-bdfb-4703-b9f8-7770e56ab2ce None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:18.496180 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:18.560075 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bd84d-8d21-4347-a8d2-f33192dc438c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:18.560683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bd84d-8d21-4347-a8d2-f33192dc438c service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:18.560957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bd84d-8d21-4347-a8d2-f33192dc438c service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:18.561264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bd84d-8d21-4347-a8d2-f33192dc438c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:18.562847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-293bd84d-8d21-4347-a8d2-f33192dc438c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:18.672575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bd84d-8d21-4347-a8d2-f33192dc438c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:18.673971 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bd84d-8d21-4347-a8d2-f33192dc438c service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:18.692142 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 568/1123] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:21.203915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7be86-ba32-4de2-b0e7-457b6daf1999 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:21.204421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7be86-ba32-4de2-b0e7-457b6daf1999 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:21.204421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7be86-ba32-4de2-b0e7-457b6daf1999 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:21.205766 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 556/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:21.211087 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758396-dfc3-4864-8af5-ad526d3f15e8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:21.211543 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758396-dfc3-4864-8af5-ad526d3f15e8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:21.211652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758396-dfc3-4864-8af5-ad526d3f15e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:21.260346 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:21.279244 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ea758396-dfc3-4864-8af5-ad526d3f15e8 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:21.280468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ea758396-dfc3-4864-8af5-ad526d3f15e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:21.377447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 569/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:21.664359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7be86-ba32-4de2-b0e7-457b6daf1999 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:21.666003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1cd7be86-ba32-4de2-b0e7-457b6daf1999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:21.770754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7be86-ba32-4de2-b0e7-457b6daf1999 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:21.812272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:21.845869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3067c0e-b178-44c6-9f77-3ad6e8b9c9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:21.846476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3067c0e-b178-44c6-9f77-3ad6e8b9c9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:21.846752 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3067c0e-b178-44c6-9f77-3ad6e8b9c9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:21.847021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3067c0e-b178-44c6-9f77-3ad6e8b9c9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:21.849253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e3067c0e-b178-44c6-9f77-3ad6e8b9c9d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:21.954872 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3067c0e-b178-44c6-9f77-3ad6e8b9c9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:21.956292 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3067c0e-b178-44c6-9f77-3ad6e8b9c9d0 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:21.976199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 557/1126] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:24.257839 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7426246-1e51-439c-84b3-05d7d5dbe79a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:24.258420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7426246-1e51-439c-84b3-05d7d5dbe79a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:24.258420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7426246-1e51-439c-84b3-05d7d5dbe79a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:24.259731 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 570/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:24.265155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e00085-aa40-4e70-86cd-c5f5ae9a3e5a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:24.265572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e00085-aa40-4e70-86cd-c5f5ae9a3e5a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:24.265736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e00085-aa40-4e70-86cd-c5f5ae9a3e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:24.331308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-82e00085-aa40-4e70-86cd-c5f5ae9a3e5a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:24.333003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-82e00085-aa40-4e70-86cd-c5f5ae9a3e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:24.353523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:24.428489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 558/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:24.714346 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7426246-1e51-439c-84b3-05d7d5dbe79a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:24.715873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d7426246-1e51-439c-84b3-05d7d5dbe79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:24.816072 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7426246-1e51-439c-84b3-05d7d5dbe79a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:24.836382 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:24.889330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85c16c-3d00-4601-ab6e-e01c1ba3f233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:24.889936 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85c16c-3d00-4601-ab6e-e01c1ba3f233 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:24.890216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85c16c-3d00-4601-ab6e-e01c1ba3f233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:24.890491 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85c16c-3d00-4601-ab6e-e01c1ba3f233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:24.892075 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9f85c16c-3d00-4601-ab6e-e01c1ba3f233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:24.999338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85c16c-3d00-4601-ab6e-e01c1ba3f233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:25.000811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85c16c-3d00-4601-ab6e-e01c1ba3f233 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:25.018775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 571/1129] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 12:57:27.242948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dac65-0a44-4231-a1a4-4826b7d1d272 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:27.243473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dac65-0a44-4231-a1a4-4826b7d1d272 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:27.243473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dac65-0a44-4231-a1a4-4826b7d1d272 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:27.244754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 559/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:27.250171 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b999bb9-0fb3-49c5-845e-31d8f5cae5c2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:27.250607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b999bb9-0fb3-49c5-845e-31d8f5cae5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:27.250736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b999bb9-0fb3-49c5-845e-31d8f5cae5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:27.314565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:27.317594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6b999bb9-0fb3-49c5-845e-31d8f5cae5c2 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:27.318830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6b999bb9-0fb3-49c5-845e-31d8f5cae5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:27.414087 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 572/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:27.697211 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dac65-0a44-4231-a1a4-4826b7d1d272 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:27.698783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1b2dac65-0a44-4231-a1a4-4826b7d1d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:27.801056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dac65-0a44-4231-a1a4-4826b7d1d272 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:27.850537 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:27.873885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63678434-614d-4063-ad8a-0bf2abb1b737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:27.874490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63678434-614d-4063-ad8a-0bf2abb1b737 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:27.874767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63678434-614d-4063-ad8a-0bf2abb1b737 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:27.875039 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63678434-614d-4063-ad8a-0bf2abb1b737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:27.876608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-63678434-614d-4063-ad8a-0bf2abb1b737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:27.983660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63678434-614d-4063-ad8a-0bf2abb1b737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:27.985096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63678434-614d-4063-ad8a-0bf2abb1b737 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:28.003653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 560/1132] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:30.208892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159ccb4-6345-48cb-b119-8463cd3dbe75 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:30.209488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159ccb4-6345-48cb-b119-8463cd3dbe75 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:30.209488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159ccb4-6345-48cb-b119-8463cd3dbe75 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:30.210843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 573/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:30.216384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9edf4-54c5-45b8-8af7-58fd2a58ba54 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:30.216803 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9edf4-54c5-45b8-8af7-58fd2a58ba54 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:30.216942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9edf4-54c5-45b8-8af7-58fd2a58ba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:30.283257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c7e9edf4-54c5-45b8-8af7-58fd2a58ba54 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:30.284495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c7e9edf4-54c5-45b8-8af7-58fd2a58ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:30.313426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:30.379961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 561/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:30.663357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159ccb4-6345-48cb-b119-8463cd3dbe75 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:30.665001 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1159ccb4-6345-48cb-b119-8463cd3dbe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:30.766737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159ccb4-6345-48cb-b119-8463cd3dbe75 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:30.796030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:30.842519 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db169258-e6bc-4cd4-8153-530ac66925f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:30.843079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db169258-e6bc-4cd4-8153-530ac66925f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:30.843394 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db169258-e6bc-4cd4-8153-530ac66925f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:30.843677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db169258-e6bc-4cd4-8153-530ac66925f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:30.845336 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db169258-e6bc-4cd4-8153-530ac66925f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:30.958757 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db169258-e6bc-4cd4-8153-530ac66925f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:30.960218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db169258-e6bc-4cd4-8153-530ac66925f4 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:30.977962 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 574/1135] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:33.194158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cf727-e127-4723-b885-e4664248b1d4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:33.194702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cf727-e127-4723-b885-e4664248b1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:33.194702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cf727-e127-4723-b885-e4664248b1d4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:33.195950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 562/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:33.201243 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f8dc4-82ec-4eeb-94d2-1af92f190222 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:33.201700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f8dc4-82ec-4eeb-94d2-1af92f190222 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:33.201830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f8dc4-82ec-4eeb-94d2-1af92f190222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:33.267663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3c1f8dc4-82ec-4eeb-94d2-1af92f190222 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:33.268876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3c1f8dc4-82ec-4eeb-94d2-1af92f190222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:33.277791 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:33.364613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 575/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:33.652717 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cf727-e127-4723-b885-e4664248b1d4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:33.654321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1f1cf727-e127-4723-b885-e4664248b1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:33.758371 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cf727-e127-4723-b885-e4664248b1d4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:33.817124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:33.836568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f9565-d9f0-4ec5-a2a3-af03fc54d421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:33.837171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f9565-d9f0-4ec5-a2a3-af03fc54d421 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:33.837477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f9565-d9f0-4ec5-a2a3-af03fc54d421 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:33.837743 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f9565-d9f0-4ec5-a2a3-af03fc54d421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:33.839322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0e8f9565-d9f0-4ec5-a2a3-af03fc54d421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:33.946599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f9565-d9f0-4ec5-a2a3-af03fc54d421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:33.948020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f9565-d9f0-4ec5-a2a3-af03fc54d421 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:33.965850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 563/1138] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:36.147548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8adbb3-be81-439d-9583-6f48e52b6689 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:36.148116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8adbb3-be81-439d-9583-6f48e52b6689 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:36.148116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8adbb3-be81-439d-9583-6f48e52b6689 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:36.149541 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 576/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:36.154858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275e8bc-d9e4-428a-a499-9c091f2d6caa None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:36.155298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275e8bc-d9e4-428a-a499-9c091f2d6caa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:36.155394 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275e8bc-d9e4-428a-a499-9c091f2d6caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:36.226710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b275e8bc-d9e4-428a-a499-9c091f2d6caa None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:36.227931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b275e8bc-d9e4-428a-a499-9c091f2d6caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:36.267345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:36.327163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 564/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:36.607271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8adbb3-be81-439d-9583-6f48e52b6689 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:36.608803 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7a8adbb3-be81-439d-9583-6f48e52b6689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:36.712486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8adbb3-be81-439d-9583-6f48e52b6689 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:36.750018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:36.784933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382a5f4-da5e-443f-8c27-d8fd33832395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:36.786351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382a5f4-da5e-443f-8c27-d8fd33832395 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:36.786666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382a5f4-da5e-443f-8c27-d8fd33832395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:36.787143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382a5f4-da5e-443f-8c27-d8fd33832395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:36.789134 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9382a5f4-da5e-443f-8c27-d8fd33832395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:36.896905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382a5f4-da5e-443f-8c27-d8fd33832395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:36.898393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382a5f4-da5e-443f-8c27-d8fd33832395 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:36.918144 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 577/1141] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:39.078248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cb5b5-08f7-4ce6-8439-ec493508484e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:39.078809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cb5b5-08f7-4ce6-8439-ec493508484e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:39.078809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cb5b5-08f7-4ce6-8439-ec493508484e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:39.080146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 565/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:39.086138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919860c-e21d-4d3e-9eb5-c386df78c299 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:39.086618 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919860c-e21d-4d3e-9eb5-c386df78c299 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:39.086714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919860c-e21d-4d3e-9eb5-c386df78c299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:39.153385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4919860c-e21d-4d3e-9eb5-c386df78c299 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:39.154533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4919860c-e21d-4d3e-9eb5-c386df78c299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:39.173650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:39.252848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 578/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:39.543185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cb5b5-08f7-4ce6-8439-ec493508484e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:39.545001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4c1cb5b5-08f7-4ce6-8439-ec493508484e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:39.649874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cb5b5-08f7-4ce6-8439-ec493508484e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:39.717237 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:39.724494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b3a46-9de9-4224-b240-ad824bceebb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:39.725104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b3a46-9de9-4224-b240-ad824bceebb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:39.725420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b3a46-9de9-4224-b240-ad824bceebb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:39.725692 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b3a46-9de9-4224-b240-ad824bceebb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:39.727294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-951b3a46-9de9-4224-b240-ad824bceebb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:39.837568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b3a46-9de9-4224-b240-ad824bceebb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:39.838907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b3a46-9de9-4224-b240-ad824bceebb2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:39.857070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 566/1144] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:42.076273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7dd38-f378-4dc6-8522-a13bb83b76ec None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:42.076897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7dd38-f378-4dc6-8522-a13bb83b76ec None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:42.076897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7dd38-f378-4dc6-8522-a13bb83b76ec None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:42.078214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 579/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 12:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:57:42.083521 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5317e-d977-42ca-ad44-b337c36d537b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:42.083932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5317e-d977-42ca-ad44-b337c36d537b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:42.084077 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5317e-d977-42ca-ad44-b337c36d537b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:42.153744 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-39d5317e-d977-42ca-ad44-b337c36d537b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:57:42.154968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-39d5317e-d977-42ca-ad44-b337c36d537b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:42.203049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:42.251024 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 567/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 12:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:57:42.532602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7dd38-f378-4dc6-8522-a13bb83b76ec None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:57:42.534193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8ea7dd38-f378-4dc6-8522-a13bb83b76ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:42.638813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7dd38-f378-4dc6-8522-a13bb83b76ec None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:57:42.686227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:42.711217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf85165-7f87-4909-9958-4883f31d5c96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:57:42.711822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf85165-7f87-4909-9958-4883f31d5c96 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:57:42.712102 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf85165-7f87-4909-9958-4883f31d5c96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:57:42.712369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf85165-7f87-4909-9958-4883f31d5c96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:57:42.713988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fcf85165-7f87-4909-9958-4883f31d5c96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:57:42.819942 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf85165-7f87-4909-9958-4883f31d5c96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:57:42.821563 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf85165-7f87-4909-9958-4883f31d5c96 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:57:42.841087 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 580/1147] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:57:52.249499 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=22 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:57:52.838995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=66 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:03.000366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811b33-eb3c-4fee-b62e-5c520303f855 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:03.000952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811b33-eb3c-4fee-b62e-5c520303f855 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:03.000952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811b33-eb3c-4fee-b62e-5c520303f855 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:03.002287 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 568/1148] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:58:03.008214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ef2e-1aac-4078-ba17-2f1921f767d3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:03.008685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ef2e-1aac-4078-ba17-2f1921f767d3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:03.008793 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ef2e-1aac-4078-ba17-2f1921f767d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:03.073382 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0996ef2e-1aac-4078-ba17-2f1921f767d3 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:58:03.074585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0996ef2e-1aac-4078-ba17-2f1921f767d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:03.170288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 581/1149] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:58:03.469314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1c021-2a3a-4e77-84ab-aa5f2fe89c14 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:03.469601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1c021-2a3a-4e77-84ab-aa5f2fe89c14 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:03.469879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1c021-2a3a-4e77-84ab-aa5f2fe89c14 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:03.471191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 569/1150] 10.222.0.30 () {54 vars in 962 bytes} [Sat Oct 15 12:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:58:03.476525 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134dfa-a45e-4c56-89da-f9544c493917 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:03.476808 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134dfa-a45e-4c56-89da-f9544c493917 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:03.477086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134dfa-a45e-4c56-89da-f9544c493917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:03.551750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-60134dfa-a45e-4c56-89da-f9544c493917 None None] MFA Rules not processed for user `5fc4d28cded84afba73b053d18d4aa23`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:58:03.552961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-60134dfa-a45e-4c56-89da-f9544c493917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:03.574365 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:03.632369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 582/1151] 10.222.0.30 () {58 vars in 1059 bytes} [Sat Oct 15 12:58:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:58:03.640388 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1c021-2a3a-4e77-84ab-aa5f2fe89c14 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:58:03.641966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f3e1c021-2a3a-4e77-84ab-aa5f2fe89c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:03.746094 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1c021-2a3a-4e77-84ab-aa5f2fe89c14 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:58:03.819130 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46f608-bd23-4350-aa8f-c7225f360526 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc4d28cded84afba73b053d18d4aa23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:58:03.819737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46f608-bd23-4350-aa8f-c7225f360526 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:03.820013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46f608-bd23-4350-aa8f-c7225f360526 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:03.820284 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46f608-bd23-4350-aa8f-c7225f360526 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:03.821970 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0b46f608-bd23-4350-aa8f-c7225f360526 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:03.876501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:03.930908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46f608-bd23-4350-aa8f-c7225f360526 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:58:03.932336 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46f608-bd23-4350-aa8f-c7225f360526 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:58:03.950620 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 570/1152] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:58:06.187017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6407ca-da83-40ef-bcd5-e2a94fc0bbeb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:06.187602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6407ca-da83-40ef-bcd5-e2a94fc0bbeb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:06.187602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6407ca-da83-40ef-bcd5-e2a94fc0bbeb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:06.188953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 583/1153] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:58:06.194783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741e9e3-91ab-4383-9aaf-1d679bd53d83 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:06.195168 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741e9e3-91ab-4383-9aaf-1d679bd53d83 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:06.195331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741e9e3-91ab-4383-9aaf-1d679bd53d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:06.261001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b741e9e3-91ab-4383-9aaf-1d679bd53d83 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:58:06.262240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b741e9e3-91ab-4383-9aaf-1d679bd53d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:06.357339 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 571/1154] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:58:06.632706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6407ca-da83-40ef-bcd5-e2a94fc0bbeb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:58:06.634309 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fd6407ca-da83-40ef-bcd5-e2a94fc0bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:06.759602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6407ca-da83-40ef-bcd5-e2a94fc0bbeb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:58:06.808623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:06.852983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404c519-7f5c-4c42-8018-fb521006985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:58:06.853624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404c519-7f5c-4c42-8018-fb521006985a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:06.853905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404c519-7f5c-4c42-8018-fb521006985a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:06.854178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404c519-7f5c-4c42-8018-fb521006985a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:06.854828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404c519-7f5c-4c42-8018-fb521006985a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:58:06.856691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-9404c519-7f5c-4c42-8018-fb521006985a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 12:58:06.856691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 15 12:58:06.856691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: Oct 15 12:58:06.857927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404c519-7f5c-4c42-8018-fb521006985a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:58:06.876642 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 584/1155] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 12:58:06 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 12:58:09.046915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015bc5-2fe7-4379-a583-51e408f45477 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:09.047464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015bc5-2fe7-4379-a583-51e408f45477 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:09.047581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015bc5-2fe7-4379-a583-51e408f45477 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:09.048849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 572/1156] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:58:09.054862 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d799ae-7932-474d-af45-8b50ad22a7f6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:09.055276 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d799ae-7932-474d-af45-8b50ad22a7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:09.055423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d799ae-7932-474d-af45-8b50ad22a7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:09.120703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-67d799ae-7932-474d-af45-8b50ad22a7f6 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:58:09.121938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-67d799ae-7932-474d-af45-8b50ad22a7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:09.216504 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 585/1157] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:58:09.508895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc015bc5-2fe7-4379-a583-51e408f45477 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:58:09.510474 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fc015bc5-2fe7-4379-a583-51e408f45477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:09.520635 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:09.658447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc015bc5-2fe7-4379-a583-51e408f45477 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:58:09.741965 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:09.774982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a2f51-8c4c-4cfc-80c4-f6af7c43d8e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:58:09.775523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a2f51-8c4c-4cfc-80c4-f6af7c43d8e9 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:09.775828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a2f51-8c4c-4cfc-80c4-f6af7c43d8e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:09.776103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a2f51-8c4c-4cfc-80c4-f6af7c43d8e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:09.777735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-959a2f51-8c4c-4cfc-80c4-f6af7c43d8e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:09.867325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a2f51-8c4c-4cfc-80c4-f6af7c43d8e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:58:09.868913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a2f51-8c4c-4cfc-80c4-f6af7c43d8e9 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:58:09.886410 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 573/1158] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:58:12.218192 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158de66-1a55-47f9-8d38-7f45dd5e54d2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:12.218787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158de66-1a55-47f9-8d38-7f45dd5e54d2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:12.218787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158de66-1a55-47f9-8d38-7f45dd5e54d2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:12.220131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 586/1159] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:58:12.226169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f688fe-d701-452b-a302-baa33444b2cc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:12.226623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f688fe-d701-452b-a302-baa33444b2cc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:12.226735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f688fe-d701-452b-a302-baa33444b2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:12.293098 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-24f688fe-d701-452b-a302-baa33444b2cc None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:58:12.294332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-24f688fe-d701-452b-a302-baa33444b2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:12.339846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:12.390500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 574/1160] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:58:12.678128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158de66-1a55-47f9-8d38-7f45dd5e54d2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:58:12.679687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a158de66-1a55-47f9-8d38-7f45dd5e54d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:12.702803 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:12.782489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158de66-1a55-47f9-8d38-7f45dd5e54d2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:58:12.856012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1365ecd-ff76-422a-bd52-e43609006e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:58:12.856544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1365ecd-ff76-422a-bd52-e43609006e6e service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:12.856822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1365ecd-ff76-422a-bd52-e43609006e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:12.857094 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1365ecd-ff76-422a-bd52-e43609006e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:12.858720 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a1365ecd-ff76-422a-bd52-e43609006e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:12.931664 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:12.965812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1365ecd-ff76-422a-bd52-e43609006e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:58:12.967276 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1365ecd-ff76-422a-bd52-e43609006e6e service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:58:12.986191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 587/1161] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:58:15.191224 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb92d6-4856-4901-ba6e-f3c7dfe70c55 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:15.191746 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb92d6-4856-4901-ba6e-f3c7dfe70c55 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:15.191746 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb92d6-4856-4901-ba6e-f3c7dfe70c55 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:15.193030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 575/1162] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:58:15.198228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b7807-7550-4d27-9189-bb16a8617a72 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:15.198653 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b7807-7550-4d27-9189-bb16a8617a72 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:15.198781 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b7807-7550-4d27-9189-bb16a8617a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:15.262736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-959b7807-7550-4d27-9189-bb16a8617a72 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:58:15.263936 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-959b7807-7550-4d27-9189-bb16a8617a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:15.356839 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 588/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:58:15.649502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb92d6-4856-4901-ba6e-f3c7dfe70c55 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:58:15.651048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ddfb92d6-4856-4901-ba6e-f3c7dfe70c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:15.754079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb92d6-4856-4901-ba6e-f3c7dfe70c55 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:58:15.825494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:15.826883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bb542-6239-43f6-a6c4-86cbb64a79b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:58:15.827490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bb542-6239-43f6-a6c4-86cbb64a79b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:15.827764 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bb542-6239-43f6-a6c4-86cbb64a79b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:15.828032 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bb542-6239-43f6-a6c4-86cbb64a79b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:15.829655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ab8bb542-6239-43f6-a6c4-86cbb64a79b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:15.937803 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bb542-6239-43f6-a6c4-86cbb64a79b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:58:15.939225 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bb542-6239-43f6-a6c4-86cbb64a79b6 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:58:15.957212 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 576/1164] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:58:18.322331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668959e-fb2c-410d-8907-843647b547f9 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:18.322892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668959e-fb2c-410d-8907-843647b547f9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:18.323022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668959e-fb2c-410d-8907-843647b547f9 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:18.324301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 589/1165] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:58:18.329481 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da296ec-9ffd-416c-9a33-71134a831a2d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:18.329931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da296ec-9ffd-416c-9a33-71134a831a2d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:18.330030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da296ec-9ffd-416c-9a33-71134a831a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:18.396944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9da296ec-9ffd-416c-9a33-71134a831a2d None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:58:18.398182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9da296ec-9ffd-416c-9a33-71134a831a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:18.450041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:18.493396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 577/1166] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:58:18.795360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668959e-fb2c-410d-8907-843647b547f9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:58:18.796894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b668959e-fb2c-410d-8907-843647b547f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:18.833842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:18.900130 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668959e-fb2c-410d-8907-843647b547f9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:58:18.973041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46eef5-7804-4174-91ea-7016f364ddee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:58:18.973666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46eef5-7804-4174-91ea-7016f364ddee service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:18.973944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46eef5-7804-4174-91ea-7016f364ddee service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:18.974213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46eef5-7804-4174-91ea-7016f364ddee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:18.975799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dc46eef5-7804-4174-91ea-7016f364ddee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:19.055582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:19.083597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46eef5-7804-4174-91ea-7016f364ddee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:58:19.085034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46eef5-7804-4174-91ea-7016f364ddee service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:58:19.103122 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 590/1167] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:58:21.643828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfa6e1-9b6b-4295-89fd-efcd4953fa83 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:21.644401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfa6e1-9b6b-4295-89fd-efcd4953fa83 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:21.644401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfa6e1-9b6b-4295-89fd-efcd4953fa83 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:21.645734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 578/1168] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 12:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 12:58:21.651280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac096-c339-4f35-8bdc-1225d1e5252a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:21.651746 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac096-c339-4f35-8bdc-1225d1e5252a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:21.651941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac096-c339-4f35-8bdc-1225d1e5252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:21.718474 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-348ac096-c339-4f35-8bdc-1225d1e5252a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 12:58:21.719883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-348ac096-c339-4f35-8bdc-1225d1e5252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:21.817416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 591/1169] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 12:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 12:58:22.109124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfa6e1-9b6b-4295-89fd-efcd4953fa83 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 12:58:22.110694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-91bfa6e1-9b6b-4295-89fd-efcd4953fa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:22.213670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfa6e1-9b6b-4295-89fd-efcd4953fa83 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 12:58:22.285673 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981a865-10de-42b4-836f-c73bb9ad027b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 12:58:22.286278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981a865-10de-42b4-836f-c73bb9ad027b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 12:58:22.286558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981a865-10de-42b4-836f-c73bb9ad027b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 12:58:22.286832 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981a865-10de-42b4-836f-c73bb9ad027b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 12:58:22.288423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5981a865-10de-42b4-836f-c73bb9ad027b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 12:58:22.297034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:22.398125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981a865-10de-42b4-836f-c73bb9ad027b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 12:58:22.399479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981a865-10de-42b4-836f-c73bb9ad027b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 12:58:22.417165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 579/1170] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 12:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 12:58:31.815673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=82 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 12:58:32.414924 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=50 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:00:01.717315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42703216-fa99-490c-8efe-3b7741a9d306 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:01.717919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42703216-fa99-490c-8efe-3b7741a9d306 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:01.717919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42703216-fa99-490c-8efe-3b7741a9d306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:01.797595 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-42703216-fa99-490c-8efe-3b7741a9d306 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:00:01.798790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-42703216-fa99-490c-8efe-3b7741a9d306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:00:01.899193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 592/1171] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:00:01.907697 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac621b20-c2d8-444f-8d95-05793b52a9f2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:01.908185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac621b20-c2d8-444f-8d95-05793b52a9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:01.908290 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac621b20-c2d8-444f-8d95-05793b52a9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:01.928489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ac621b20-c2d8-444f-8d95-05793b52a9f2 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:00:01.929723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ac621b20-c2d8-444f-8d95-05793b52a9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:00:01.938249 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 580/1172] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:00:01.942639 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42703216-fa99-490c-8efe-3b7741a9d306 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:01.952016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42703216-fa99-490c-8efe-3b7741a9d306 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:01.956970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:01.957598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:01.957875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:01.958146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:01.958789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:00:01.959089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:01.960096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:00:01.960096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 15 13:00:01.960096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: Oct 15 13:00:01.960977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:01.974894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 593/1173] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:00:01.984647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac621b20-c2d8-444f-8d95-05793b52a9f2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:01.993772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac621b20-c2d8-444f-8d95-05793b52a9f2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:01.998347 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561744c-bbaa-4fde-94de-0d461e9e1f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:01.998945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561744c-bbaa-4fde-94de-0d461e9e1f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:01.999226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561744c-bbaa-4fde-94de-0d461e9e1f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:01.999498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561744c-bbaa-4fde-94de-0d461e9e1f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.000285 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561744c-bbaa-4fde-94de-0d461e9e1f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:02.001313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-c561744c-bbaa-4fde-94de-0d461e9e1f0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:00:02.001313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Oct 15 13:00:02.001313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: Oct 15 13:00:02.002509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561744c-bbaa-4fde-94de-0d461e9e1f0b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:02.120112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 581/1174] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:00:01 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:00:02.122618 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:02.133086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cde6e-f029-45a2-a380-88403ddd57fb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:02.182058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:00:02.189037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edce4e8-b6e1-4821-acaf-7d88b01d7837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:02.190101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edce4e8-b6e1-4821-acaf-7d88b01d7837 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.190101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edce4e8-b6e1-4821-acaf-7d88b01d7837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.190440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edce4e8-b6e1-4821-acaf-7d88b01d7837 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.190817 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edce4e8-b6e1-4821-acaf-7d88b01d7837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:02.191831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-2edce4e8-b6e1-4821-acaf-7d88b01d7837 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:00:02.191831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 15 13:00:02.191831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: Oct 15 13:00:02.193034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edce4e8-b6e1-4821-acaf-7d88b01d7837 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:02.294443 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 594/1175] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:00:02 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:00:02.299035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561744c-bbaa-4fde-94de-0d461e9e1f0b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:02.309039 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561744c-bbaa-4fde-94de-0d461e9e1f0b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:02.313653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b22a0f-bb73-4ed3-9289-f73dcba28cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:02.314245 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b22a0f-bb73-4ed3-9289-f73dcba28cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.314523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b22a0f-bb73-4ed3-9289-f73dcba28cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.314795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b22a0f-bb73-4ed3-9289-f73dcba28cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.315429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b22a0f-bb73-4ed3-9289-f73dcba28cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:02.317016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b22a0f-bb73-4ed3-9289-f73dcba28cf4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:02.330206 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 582/1176] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:00:02.334031 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edce4e8-b6e1-4821-acaf-7d88b01d7837 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:02.344418 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edce4e8-b6e1-4821-acaf-7d88b01d7837 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:02.348956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c29f5-46e5-40a9-951e-b2be9f64a40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:02.349579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c29f5-46e5-40a9-951e-b2be9f64a40b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.349858 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c29f5-46e5-40a9-951e-b2be9f64a40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.350132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c29f5-46e5-40a9-951e-b2be9f64a40b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.350763 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c29f5-46e5-40a9-951e-b2be9f64a40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:02.352350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c29f5-46e5-40a9-951e-b2be9f64a40b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:02.365735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 595/1177] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:00:02.369542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b22a0f-bb73-4ed3-9289-f73dcba28cf4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:02.379306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b22a0f-bb73-4ed3-9289-f73dcba28cf4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:02.383869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f056c-9562-4660-8f02-5f19db16a76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:02.384506 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f056c-9562-4660-8f02-5f19db16a76f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.384784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f056c-9562-4660-8f02-5f19db16a76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.385057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f056c-9562-4660-8f02-5f19db16a76f admin admin] PATH_INFO: `/v3/projects/283e42b5daaa416ea0a78a2ac1acc587/users/e61ecbbc17124c2191776d78ca7045b5/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.399776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f056c-9562-4660-8f02-5f19db16a76f admin admin] RBAC: Authorizing `identity:create_grant(project_id=283e42b5daaa416ea0a78a2ac1acc587, user_id=e61ecbbc17124c2191776d78ca7045b5, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:02.402456 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f056c-9562-4660-8f02-5f19db16a76f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:02.471077 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 583/1178] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:00:02 2022] PUT /identity/v3/projects/283e42b5daaa416ea0a78a2ac1acc587/users/e61ecbbc17124c2191776d78ca7045b5/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:00:02.482167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e62bc1-0931-4f14-b9d0-573f6016bbae None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.482441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e62bc1-0931-4f14-b9d0-573f6016bbae None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.482722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e62bc1-0931-4f14-b9d0-573f6016bbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.510609 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a9e62bc1-0931-4f14-b9d0-573f6016bbae None None] MFA Rules not processed for user `e61ecbbc17124c2191776d78ca7045b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:00:02.511818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a9e62bc1-0931-4f14-b9d0-573f6016bbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:00:02.575266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 596/1179] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:00:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:00:02.589618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509662c1-ece8-464a-979a-c7daf30362dd None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.592234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509662c1-ece8-464a-979a-c7daf30362dd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.592234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509662c1-ece8-464a-979a-c7daf30362dd None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.592234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 584/1180] 10.222.0.30 () {54 vars in 963 bytes} [Sat Oct 15 13:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:00:02.597029 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c00e5-a350-4a26-ab50-5f59d124b2cf None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.597349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c00e5-a350-4a26-ab50-5f59d124b2cf None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.597625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c00e5-a350-4a26-ab50-5f59d124b2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.655375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-de2c00e5-a350-4a26-ab50-5f59d124b2cf None None] MFA Rules not processed for user `264bc0b947614ec2a554d627e9e2da56`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:00:02.657150 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-de2c00e5-a350-4a26-ab50-5f59d124b2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:00:02.732447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 597/1181] 10.222.0.30 () {58 vars in 1060 bytes} [Sat Oct 15 13:00:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:00:02.740420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509662c1-ece8-464a-979a-c7daf30362dd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:02.751313 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509662c1-ece8-464a-979a-c7daf30362dd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:02.754802 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2113ed-79e0-4e59-a903-29a9c68984c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:02.755395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2113ed-79e0-4e59-a903-29a9c68984c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.755678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2113ed-79e0-4e59-a903-29a9c68984c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.755968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2113ed-79e0-4e59-a903-29a9c68984c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.763974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2113ed-79e0-4e59-a903-29a9c68984c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:02.766049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2113ed-79e0-4e59-a903-29a9c68984c8 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:02.774717 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 585/1182] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:00:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:00:02.820263 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2aab-cd7e-4f86-bcc3-96a446bd33dc None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.820553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2aab-cd7e-4f86-bcc3-96a446bd33dc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.820828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2aab-cd7e-4f86-bcc3-96a446bd33dc None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.822115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 598/1183] 10.222.0.30 () {52 vars in 830 bytes} [Sat Oct 15 13:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:00:02.825655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2113ed-79e0-4e59-a903-29a9c68984c8 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:02.886804 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2113ed-79e0-4e59-a903-29a9c68984c8 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:02.892084 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:02.892705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:02.892982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:02.893287 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] PATH_INFO: `/v3/users/e61ecbbc17124c2191776d78ca7045b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:02.894431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61ecbbc17124c2191776d78ca7045b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:02.895439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:00:02.895439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Oct 15 13:00:02.895439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: Oct 15 13:00:02.896650 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:03.127982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61ecbbc17124c2191776d78ca7045b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:03.128612 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61ecbbc17124c2191776d78ca7045b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:03.143291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61ecbbc17124c2191776d78ca7045b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:03.144051 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61ecbbc17124c2191776d78ca7045b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:03.214795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61ecbbc17124c2191776d78ca7045b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:03.237673 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:00:03.246021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 586/1184] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:00:02 2022] DELETE /identity/v3/users/e61ecbbc17124c2191776d78ca7045b5 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:00:03.256207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc63f97-bc50-4a97-ba50-37b156d1dfcc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:03.256207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc63f97-bc50-4a97-ba50-37b156d1dfcc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:03.256207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc63f97-bc50-4a97-ba50-37b156d1dfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:03.271148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fdc63f97-bc50-4a97-ba50-37b156d1dfcc None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:00:03.272350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fdc63f97-bc50-4a97-ba50-37b156d1dfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:00:03.281617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:00:03.329626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 599/1185] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:00:03.339265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:03.340672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:00:03.418002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e95bc9-dd43-4b2e-ac23-df5bec6cfdb2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:03.422372 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9ca11-464b-4161-9b82-caa414e236d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:03.423070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9ca11-464b-4161-9b82-caa414e236d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:03.423351 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9ca11-464b-4161-9b82-caa414e236d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:03.423621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9ca11-464b-4161-9b82-caa414e236d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:03.433361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9ca11-464b-4161-9b82-caa414e236d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:03.437431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9ca11-464b-4161-9b82-caa414e236d5 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:03.444222 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 587/1186] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:00:03.924901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc63f97-bc50-4a97-ba50-37b156d1dfcc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:00:03.926373 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fdc63f97-bc50-4a97-ba50-37b156d1dfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:00:03.940911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc63f97-bc50-4a97-ba50-37b156d1dfcc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:00:03.945513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:00:03.946151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:00:03.946431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:00:03.946701 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] PATH_INFO: `/v3/projects/283e42b5daaa416ea0a78a2ac1acc587` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:00:03.947813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=283e42b5daaa416ea0a78a2ac1acc587)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:00:03.948838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:00:03.948838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 15 13:00:03.948838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: Oct 15 13:00:03.950125 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:00:04.066416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '283e42b5daaa416ea0a78a2ac1acc587', 'request_id': 'req-09c0179e-013b-4b3e-a7f9-428072225371'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:04.151544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '283e42b5daaa416ea0a78a2ac1acc587', 'request_id': 'req-09c0179e-013b-4b3e-a7f9-428072225371'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:04.166366 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '283e42b5daaa416ea0a78a2ac1acc587', 'request_id': 'req-09c0179e-013b-4b3e-a7f9-428072225371'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:04.177873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] The token cache is being invalidate because project 283e42b5daaa416ea0a78a2ac1acc587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:00:04.178169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:04.178826 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09c0179e-013b-4b3e-a7f9-428072225371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:00:04.180428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 600/1187] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:00:03 2022] DELETE /identity/v3/projects/283e42b5daaa416ea0a78a2ac1acc587 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:00:13.488887 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:00:14.174947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:01:29.982155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfd84c-9766-4fdb-ae46-9be8606e0ffe None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:29.982715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfd84c-9766-4fdb-ae46-9be8606e0ffe None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:29.982715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfd84c-9766-4fdb-ae46-9be8606e0ffe None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:29.984005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 588/1188] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:01:29.988927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4b9ea-4ce4-47ea-973c-1e7060367724 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:29.989390 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4b9ea-4ce4-47ea-973c-1e7060367724 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:29.989506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4b9ea-4ce4-47ea-973c-1e7060367724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:30.009490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-41e4b9ea-4ce4-47ea-973c-1e7060367724 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:30.010697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-41e4b9ea-4ce4-47ea-973c-1e7060367724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:30.069201 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 601/1189] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:01:30.226021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfd84c-9766-4fdb-ae46-9be8606e0ffe None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:01:30.227822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7adfd84c-9766-4fdb-ae46-9be8606e0ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:30.349626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfd84c-9766-4fdb-ae46-9be8606e0ffe None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:01:30.355324 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0881fc-7857-470e-86cf-f94ddafa67f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:01:30.355915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0881fc-7857-470e-86cf-f94ddafa67f9 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:30.356192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0881fc-7857-470e-86cf-f94ddafa67f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:30.356464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0881fc-7857-470e-86cf-f94ddafa67f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:30.365310 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0881fc-7857-470e-86cf-f94ddafa67f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:01:30.366895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0881fc-7857-470e-86cf-f94ddafa67f9 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:01:30.376048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 589/1190] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:01:31.465843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db34f0-8c8f-4e30-97ab-c82f5c2a1824 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:31.466351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db34f0-8c8f-4e30-97ab-c82f5c2a1824 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:31.466351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db34f0-8c8f-4e30-97ab-c82f5c2a1824 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:31.467668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 602/1191] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:01:31.474223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc7a07-91f3-4c3f-bc25-3c9f05f8c6f8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:31.474678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc7a07-91f3-4c3f-bc25-3c9f05f8c6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:31.474774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc7a07-91f3-4c3f-bc25-3c9f05f8c6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:31.496564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6acc7a07-91f3-4c3f-bc25-3c9f05f8c6f8 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:31.497817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6acc7a07-91f3-4c3f-bc25-3c9f05f8c6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:31.506466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 590/1192] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:01:31.658649 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db34f0-8c8f-4e30-97ab-c82f5c2a1824 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:01:31.672854 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db34f0-8c8f-4e30-97ab-c82f5c2a1824 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:01:31.678668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ed089-ab25-44f3-a6d8-7f94ae33675f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:01:31.679276 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ed089-ab25-44f3-a6d8-7f94ae33675f service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:31.679560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ed089-ab25-44f3-a6d8-7f94ae33675f service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:31.679837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ed089-ab25-44f3-a6d8-7f94ae33675f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:31.687973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ed089-ab25-44f3-a6d8-7f94ae33675f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:01:31.690211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ed089-ab25-44f3-a6d8-7f94ae33675f service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:01:31.700305 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 603/1193] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:01:31.760423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51914e7-29ad-4c8b-ad0e-69da8d98192a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:31.760707 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51914e7-29ad-4c8b-ad0e-69da8d98192a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:31.760987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51914e7-29ad-4c8b-ad0e-69da8d98192a None None] PATH_INFO: `/v3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:31.762384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 591/1194] 10.222.0.39 () {54 vars in 925 bytes} [Sat Oct 15 13:01:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:01:31.769570 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb59ed0-6e25-4904-af40-21e2864dd25d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:31.769975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb59ed0-6e25-4904-af40-21e2864dd25d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:31.770347 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb59ed0-6e25-4904-af40-21e2864dd25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:31.825641 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7bb59ed0-6e25-4904-af40-21e2864dd25d None None] MFA Rules not processed for user `18df1f401bb2494299eda9582c85a04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:31.826890 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7bb59ed0-6e25-4904-af40-21e2864dd25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:31.898612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 604/1195] 10.222.0.39 () {58 vars in 1015 bytes} [Sat Oct 15 13:01:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:01:40.715880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa5c20-d736-4979-ae2a-941a82c5d34c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:40.716330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa5c20-d736-4979-ae2a-941a82c5d34c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:40.716434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa5c20-d736-4979-ae2a-941a82c5d34c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:40.717861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 592/1196] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 13:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:01:40.723104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ec7ad-197d-40cd-a7e0-f29fe09bf232 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:40.723512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ec7ad-197d-40cd-a7e0-f29fe09bf232 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:40.723662 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ec7ad-197d-40cd-a7e0-f29fe09bf232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:40.742967 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ea5ec7ad-197d-40cd-a7e0-f29fe09bf232 None None] MFA Rules not processed for user `18df1f401bb2494299eda9582c85a04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:40.744182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ea5ec7ad-197d-40cd-a7e0-f29fe09bf232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:40.753362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 605/1197] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 13:01:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:01:40.760923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa5c20-d736-4979-ae2a-941a82c5d34c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:01:40.774847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa5c20-d736-4979-ae2a-941a82c5d34c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:01:40.779813 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494b1b0-0af8-42d5-a594-d78167273667 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18df1f401bb2494299eda9582c85a04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:01:40.780413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494b1b0-0af8-42d5-a594-d78167273667 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:40.780699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494b1b0-0af8-42d5-a594-d78167273667 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:40.780978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494b1b0-0af8-42d5-a594-d78167273667 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:40.789421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494b1b0-0af8-42d5-a594-d78167273667 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:01:40.791009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494b1b0-0af8-42d5-a594-d78167273667 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:01:40.801428 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 593/1198] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:01:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:01:43.177530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d208-cb1e-4da5-b4c8-6380fa7235da None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:43.178101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d208-cb1e-4da5-b4c8-6380fa7235da None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:43.178186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d208-cb1e-4da5-b4c8-6380fa7235da None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:43.179627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 606/1199] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:01:43.186536 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768649f-546c-42c2-a059-e82f0844ebc2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:43.187050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768649f-546c-42c2-a059-e82f0844ebc2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:43.187156 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768649f-546c-42c2-a059-e82f0844ebc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:43.208539 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c768649f-546c-42c2-a059-e82f0844ebc2 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:43.209773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c768649f-546c-42c2-a059-e82f0844ebc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:43.217953 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 594/1200] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:01:43.378640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07d208-cb1e-4da5-b4c8-6380fa7235da None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:01:43.392710 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07d208-cb1e-4da5-b4c8-6380fa7235da None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:01:43.398386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33720b83-5994-4ce5-a4b1-2c2e9c867af9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:01:43.398961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33720b83-5994-4ce5-a4b1-2c2e9c867af9 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:43.399240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33720b83-5994-4ce5-a4b1-2c2e9c867af9 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:43.399511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33720b83-5994-4ce5-a4b1-2c2e9c867af9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:43.407594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33720b83-5994-4ce5-a4b1-2c2e9c867af9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:01:43.409893 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33720b83-5994-4ce5-a4b1-2c2e9c867af9 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:01:43.419332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 607/1201] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:01:44.487796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a80790-89d2-4bca-87dd-ad2d923d5314 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:44.488370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a80790-89d2-4bca-87dd-ad2d923d5314 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:44.488370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a80790-89d2-4bca-87dd-ad2d923d5314 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:44.489766 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 595/1202] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:01:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:01:44.496121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2230a-9c3d-4c43-bba6-fe5fc2706f3a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:44.496548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2230a-9c3d-4c43-bba6-fe5fc2706f3a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:44.496713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2230a-9c3d-4c43-bba6-fe5fc2706f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:44.516310 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-54d2230a-9c3d-4c43-bba6-fe5fc2706f3a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:44.521098 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-54d2230a-9c3d-4c43-bba6-fe5fc2706f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:44.525869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 608/1203] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:01:44.678164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a80790-89d2-4bca-87dd-ad2d923d5314 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:01:44.693472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a80790-89d2-4bca-87dd-ad2d923d5314 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:01:44.698943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e549ad-0458-4606-b629-4c5b3cb23a44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:01:44.699539 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e549ad-0458-4606-b629-4c5b3cb23a44 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:44.699821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e549ad-0458-4606-b629-4c5b3cb23a44 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:44.700097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e549ad-0458-4606-b629-4c5b3cb23a44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:44.708447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e549ad-0458-4606-b629-4c5b3cb23a44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:01:44.710055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e549ad-0458-4606-b629-4c5b3cb23a44 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:01:44.719227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 596/1204] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:01:48.659020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402024ff-8171-4259-993d-107203ae6c9d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:48.659620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402024ff-8171-4259-993d-107203ae6c9d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:48.659620 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402024ff-8171-4259-993d-107203ae6c9d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:48.660980 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 609/1205] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:01:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:01:48.666172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa3ffc-1a0e-42d6-8d1c-5d5f7b2fde6e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:48.666618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa3ffc-1a0e-42d6-8d1c-5d5f7b2fde6e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:48.666736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa3ffc-1a0e-42d6-8d1c-5d5f7b2fde6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:48.686663 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-69aa3ffc-1a0e-42d6-8d1c-5d5f7b2fde6e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:48.687864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-69aa3ffc-1a0e-42d6-8d1c-5d5f7b2fde6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:48.695783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 597/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:01:48.986233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402024ff-8171-4259-993d-107203ae6c9d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:01:48.988535 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-402024ff-8171-4259-993d-107203ae6c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:49.016821 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:01:49.046008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402024ff-8171-4259-993d-107203ae6c9d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:01:49.050831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd89ab-04a1-4275-a878-d37459de248f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:01:49.051434 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd89ab-04a1-4275-a878-d37459de248f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:49.051716 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd89ab-04a1-4275-a878-d37459de248f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:49.051991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd89ab-04a1-4275-a878-d37459de248f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:49.060834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd89ab-04a1-4275-a878-d37459de248f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:01:49.062324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd89ab-04a1-4275-a878-d37459de248f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:01:49.071850 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 610/1207] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:01:51.233861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c85c8b-d672-4fb6-9305-1ad8a3543166 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:51.234416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c85c8b-d672-4fb6-9305-1ad8a3543166 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:51.234416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c85c8b-d672-4fb6-9305-1ad8a3543166 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:51.235786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 598/1208] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:01:51.241054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b039-3767-42ae-a278-0b1e02426151 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:51.241493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b039-3767-42ae-a278-0b1e02426151 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:51.241633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b039-3767-42ae-a278-0b1e02426151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:51.262091 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1a17b039-3767-42ae-a278-0b1e02426151 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:51.263319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1a17b039-3767-42ae-a278-0b1e02426151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:51.271810 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 611/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:01:51.560123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c85c8b-d672-4fb6-9305-1ad8a3543166 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:01:51.571090 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c85c8b-d672-4fb6-9305-1ad8a3543166 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:01:51.574738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e4720-d0c1-4b60-a977-bfe6dd99e58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:01:51.575334 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e4720-d0c1-4b60-a977-bfe6dd99e58f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:51.575612 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e4720-d0c1-4b60-a977-bfe6dd99e58f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:51.575883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e4720-d0c1-4b60-a977-bfe6dd99e58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:51.585681 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e4720-d0c1-4b60-a977-bfe6dd99e58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:01:51.587103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e4720-d0c1-4b60-a977-bfe6dd99e58f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:01:51.597011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 599/1210] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:01:55.220174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f0fa3-b2c3-43f9-8921-741b74a3bb07 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:55.220771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f0fa3-b2c3-43f9-8921-741b74a3bb07 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:55.220771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f0fa3-b2c3-43f9-8921-741b74a3bb07 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:55.222315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 612/1211] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:01:55.227453 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda66e5-be93-4dba-82cc-6962ec251f3d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:55.227850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda66e5-be93-4dba-82cc-6962ec251f3d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:55.228008 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda66e5-be93-4dba-82cc-6962ec251f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:55.235542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=74 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:01:55.248375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5bda66e5-be93-4dba-82cc-6962ec251f3d None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:55.251448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5bda66e5-be93-4dba-82cc-6962ec251f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:55.258183 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 600/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:01:55.565511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f0fa3-b2c3-43f9-8921-741b74a3bb07 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:01:55.567201 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-342f0fa3-b2c3-43f9-8921-741b74a3bb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:55.625249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f0fa3-b2c3-43f9-8921-741b74a3bb07 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:01:55.628924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852c8f1-35b6-4055-ae05-74338e3128ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:01:55.629569 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852c8f1-35b6-4055-ae05-74338e3128ac service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:55.629848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852c8f1-35b6-4055-ae05-74338e3128ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:55.630120 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852c8f1-35b6-4055-ae05-74338e3128ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:55.638863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852c8f1-35b6-4055-ae05-74338e3128ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:01:55.640316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852c8f1-35b6-4055-ae05-74338e3128ac service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:01:55.649976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 613/1213] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:01:57.918373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bfa58-0027-4ca6-b7b6-1dcbea78445e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:57.918920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bfa58-0027-4ca6-b7b6-1dcbea78445e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:57.918920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bfa58-0027-4ca6-b7b6-1dcbea78445e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:57.920221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 601/1214] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:01:57.926364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600bd76-157b-4514-b6e5-c58e9ab3520f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:57.926751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600bd76-157b-4514-b6e5-c58e9ab3520f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:57.926917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600bd76-157b-4514-b6e5-c58e9ab3520f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:57.947607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5600bd76-157b-4514-b6e5-c58e9ab3520f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:01:57.948833 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5600bd76-157b-4514-b6e5-c58e9ab3520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:57.956683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 614/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:01:58.259454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bfa58-0027-4ca6-b7b6-1dcbea78445e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:01:58.261113 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e60bfa58-0027-4ca6-b7b6-1dcbea78445e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:01:58.275168 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bfa58-0027-4ca6-b7b6-1dcbea78445e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:01:58.278716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b1745-e673-4dca-a9af-ee042d68c256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:01:58.279312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b1745-e673-4dca-a9af-ee042d68c256 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:01:58.279593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b1745-e673-4dca-a9af-ee042d68c256 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:01:58.279866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b1745-e673-4dca-a9af-ee042d68c256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:01:58.289918 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b1745-e673-4dca-a9af-ee042d68c256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:01:58.291332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b1745-e673-4dca-a9af-ee042d68c256 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:01:58.301364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 602/1216] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:02:07.942875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=47 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:02:08.296476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=8 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:22.448486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ca026-35ca-4b14-95f5-bc7cce358ab2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.449139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ca026-35ca-4b14-95f5-bc7cce358ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.449139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ca026-35ca-4b14-95f5-bc7cce358ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.474429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-526ca026-35ca-4b14-95f5-bc7cce358ab2 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:22.475687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-526ca026-35ca-4b14-95f5-bc7cce358ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:22.490363 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 615/1217] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:22.501082 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f2f15-3770-4348-ab7c-de0f0db51acc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.501711 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f2f15-3770-4348-ab7c-de0f0db51acc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.501711 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f2f15-3770-4348-ab7c-de0f0db51acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.544366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-984f2f15-3770-4348-ab7c-de0f0db51acc None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:22.545683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-984f2f15-3770-4348-ab7c-de0f0db51acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:22.567505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 603/1218] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:22.572583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ca026-35ca-4b14-95f5-bc7cce358ab2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:22.583151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ca026-35ca-4b14-95f5-bc7cce358ab2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:22.594118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88770401-221a-45d4-993a-6231ba16228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:22.595104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88770401-221a-45d4-993a-6231ba16228d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.595603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88770401-221a-45d4-993a-6231ba16228d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.596114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88770401-221a-45d4-993a-6231ba16228d admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.597947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88770401-221a-45d4-993a-6231ba16228d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:22.598501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88770401-221a-45d4-993a-6231ba16228d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:22.600465 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88770401-221a-45d4-993a-6231ba16228d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:22.622509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 616/1219] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:22.631657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f2f15-3770-4348-ab7c-de0f0db51acc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:22.642761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f2f15-3770-4348-ab7c-de0f0db51acc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:22.655536 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb9cc0-ff7e-4f77-95d2-e2cbd73f6b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:22.656185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb9cc0-ff7e-4f77-95d2-e2cbd73f6b76 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.656465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb9cc0-ff7e-4f77-95d2-e2cbd73f6b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.656748 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb9cc0-ff7e-4f77-95d2-e2cbd73f6b76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.657417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb9cc0-ff7e-4f77-95d2-e2cbd73f6b76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:22.659988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb9cc0-ff7e-4f77-95d2-e2cbd73f6b76 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:22.691145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd90c1-64b5-408f-8e62-35347ce5a535 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.691774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd90c1-64b5-408f-8e62-35347ce5a535 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.693157 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd90c1-64b5-408f-8e62-35347ce5a535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.731310 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-70dd90c1-64b5-408f-8e62-35347ce5a535 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:22.731310 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-70dd90c1-64b5-408f-8e62-35347ce5a535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:22.741528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 617/1220] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:03:22.743394 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95aa9f-95e2-436c-b0f5-84f093cab35f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.744005 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95aa9f-95e2-436c-b0f5-84f093cab35f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.744512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95aa9f-95e2-436c-b0f5-84f093cab35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.776195 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ed95aa9f-95e2-436c-b0f5-84f093cab35f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:22.777912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ed95aa9f-95e2-436c-b0f5-84f093cab35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:22.796009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 618/1221] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:22.797662 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448b84-dba4-44ea-ab24-5761b4715210 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.798174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448b84-dba4-44ea-ab24-5761b4715210 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.798777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448b84-dba4-44ea-ab24-5761b4715210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.805703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 604/1222] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:22.809674 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb9cc0-ff7e-4f77-95d2-e2cbd73f6b76 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:22.832645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-24448b84-dba4-44ea-ab24-5761b4715210 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:22.833332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-24448b84-dba4-44ea-ab24-5761b4715210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:22.917214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb9cc0-ff7e-4f77-95d2-e2cbd73f6b76 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:22.921610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 619/1223] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:22.923056 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa49de-b18d-4aa7-998f-efea965598dc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.923354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa49de-b18d-4aa7-998f-efea965598dc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.923637 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa49de-b18d-4aa7-998f-efea965598dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.926830 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342e068-0321-4ef3-bbf7-a91f585b9f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:22.927465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e068-0321-4ef3-bbf7-a91f585b9f52 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.927748 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e068-0321-4ef3-bbf7-a91f585b9f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.928025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e068-0321-4ef3-bbf7-a91f585b9f52 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.928731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342e068-0321-4ef3-bbf7-a91f585b9f52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:22.932623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-2342e068-0321-4ef3-bbf7-a91f585b9f52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:03:22.932623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Oct 15 13:03:22.932623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: Oct 15 13:03:22.937873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342e068-0321-4ef3-bbf7-a91f585b9f52 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:22.947856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-affa49de-b18d-4aa7-998f-efea965598dc None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:22.949116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-affa49de-b18d-4aa7-998f-efea965598dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:22.957254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 620/1224] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:22.960604 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af222d19-3f62-45c3-af3f-348bd07dd530 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.960604 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af222d19-3f62-45c3-af3f-348bd07dd530 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.960604 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af222d19-3f62-45c3-af3f-348bd07dd530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:22.978046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-af222d19-3f62-45c3-af3f-348bd07dd530 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:22.979241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-af222d19-3f62-45c3-af3f-348bd07dd530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:22.989285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 621/1225] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:22.989285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d3273-c43a-400c-b288-4579f8de6d06 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:22.989785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d3273-c43a-400c-b288-4579f8de6d06 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:22.989785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d3273-c43a-400c-b288-4579f8de6d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.008862 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fc7d3273-c43a-400c-b288-4579f8de6d06 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:23.010076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fc7d3273-c43a-400c-b288-4579f8de6d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:23.017817 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 622/1226] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:23.020328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa5881-7207-476b-aa4d-777477d78aeb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.020328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa5881-7207-476b-aa4d-777477d78aeb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.020328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa5881-7207-476b-aa4d-777477d78aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.038399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e2fa5881-7207-476b-aa4d-777477d78aeb None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:23.039590 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e2fa5881-7207-476b-aa4d-777477d78aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:23.047403 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 623/1227] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:23.048174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa5881-7207-476b-aa4d-777477d78aeb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.063093 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 605/1228] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:22 2022] POST /identity/v3/users => generated 294 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:03:23.063093 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce65708-bc45-49c1-ba38-cdf9a7b6f794 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.063093 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce65708-bc45-49c1-ba38-cdf9a7b6f794 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.064070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa5881-7207-476b-aa4d-777477d78aeb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.064166 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce65708-bc45-49c1-ba38-cdf9a7b6f794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.075815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a5179-32cd-43cb-83ea-170a958b6a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.075815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a5179-32cd-43cb-83ea-170a958b6a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.075815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a5179-32cd-43cb-83ea-170a958b6a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.075815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a5179-32cd-43cb-83ea-170a958b6a0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.076160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a5179-32cd-43cb-83ea-170a958b6a0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:23.076160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a5179-32cd-43cb-83ea-170a958b6a0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.076160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a5179-32cd-43cb-83ea-170a958b6a0a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.083159 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cce65708-bc45-49c1-ba38-cdf9a7b6f794 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:23.083830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 624/1229] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:23.084485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cce65708-bc45-49c1-ba38-cdf9a7b6f794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:23.084906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29838d2d-b770-477c-8b47-300b1077ac68 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.085219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29838d2d-b770-477c-8b47-300b1077ac68 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.085502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29838d2d-b770-477c-8b47-300b1077ac68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.099151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 606/1230] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:23.100295 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce65708-bc45-49c1-ba38-cdf9a7b6f794 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.106717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-29838d2d-b770-477c-8b47-300b1077ac68 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:23.107949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-29838d2d-b770-477c-8b47-300b1077ac68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:23.111339 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce65708-bc45-49c1-ba38-cdf9a7b6f794 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.116535 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f5c45-890a-4ea7-b0f7-249f9a63bdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.121508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 625/1231] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:23.121619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f5c45-890a-4ea7-b0f7-249f9a63bdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.121619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f5c45-890a-4ea7-b0f7-249f9a63bdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.121619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f5c45-890a-4ea7-b0f7-249f9a63bdf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.121851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d727bc4-887c-4233-8a06-fb111e82f464 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.121851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d727bc4-887c-4233-8a06-fb111e82f464 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.121997 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f5c45-890a-4ea7-b0f7-249f9a63bdf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:23.122094 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d727bc4-887c-4233-8a06-fb111e82f464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.122252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f5c45-890a-4ea7-b0f7-249f9a63bdf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.122252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-656f5c45-890a-4ea7-b0f7-249f9a63bdf0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:03:23.122252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Oct 15 13:03:23.122252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: Oct 15 13:03:23.122252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f5c45-890a-4ea7-b0f7-249f9a63bdf0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.132735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 607/1232] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:23.138687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c3054-5803-4c56-ae8b-3fe08726fce0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.138981 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c3054-5803-4c56-ae8b-3fe08726fce0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.139264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c3054-5803-4c56-ae8b-3fe08726fce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.142081 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4d727bc4-887c-4233-8a06-fb111e82f464 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:23.143345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4d727bc4-887c-4233-8a06-fb111e82f464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:23.151732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 626/1233] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:23.155879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622cc40-53b1-453d-a2eb-36b650edf6e2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.155879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622cc40-53b1-453d-a2eb-36b650edf6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.155879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622cc40-53b1-453d-a2eb-36b650edf6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.160431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-959c3054-5803-4c56-ae8b-3fe08726fce0 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:23.161654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-959c3054-5803-4c56-ae8b-3fe08726fce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:23.169532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 608/1234] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:23.170448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c3054-5803-4c56-ae8b-3fe08726fce0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.178760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f622cc40-53b1-453d-a2eb-36b650edf6e2 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:23.179982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f622cc40-53b1-453d-a2eb-36b650edf6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:23.181724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c3054-5803-4c56-ae8b-3fe08726fce0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.186256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d222f4-ffbf-4575-ad65-1521133436d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.186851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d222f4-ffbf-4575-ad65-1521133436d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.187131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d222f4-ffbf-4575-ad65-1521133436d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.187416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d222f4-ffbf-4575-ad65-1521133436d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.188035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d222f4-ffbf-4575-ad65-1521133436d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.188184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 627/1235] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:23.188928 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622cc40-53b1-453d-a2eb-36b650edf6e2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.189738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d222f4-ffbf-4575-ad65-1521133436d4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.199636 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622cc40-53b1-453d-a2eb-36b650edf6e2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.200860 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 609/1236] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:23.202260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a152ab-b834-4c49-b13a-37d4f30bd49b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.202554 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a152ab-b834-4c49-b13a-37d4f30bd49b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.202827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a152ab-b834-4c49-b13a-37d4f30bd49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.204705 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38509610-fb35-44b4-b7c2-49179ea79c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.205352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38509610-fb35-44b4-b7c2-49179ea79c47 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.205697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38509610-fb35-44b4-b7c2-49179ea79c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.206114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38509610-fb35-44b4-b7c2-49179ea79c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.206948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38509610-fb35-44b4-b7c2-49179ea79c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.207982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-38509610-fb35-44b4-b7c2-49179ea79c47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:03:23.207982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 15 13:03:23.207982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: Oct 15 13:03:23.209211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38509610-fb35-44b4-b7c2-49179ea79c47 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.224069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-04a152ab-b834-4c49-b13a-37d4f30bd49b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:23.229511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-04a152ab-b834-4c49-b13a-37d4f30bd49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:23.233251 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 610/1237] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:23.234441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5362f50-a23a-45c7-a703-609352e87d79 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.234728 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5362f50-a23a-45c7-a703-609352e87d79 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.235010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5362f50-a23a-45c7-a703-609352e87d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.255002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a5362f50-a23a-45c7-a703-609352e87d79 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:23.256135 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a5362f50-a23a-45c7-a703-609352e87d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:23.263976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 611/1238] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:23.264732 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5362f50-a23a-45c7-a703-609352e87d79 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.274103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5362f50-a23a-45c7-a703-609352e87d79 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.278666 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403f890-81f3-4746-82aa-adf556c98ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.287103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403f890-81f3-4746-82aa-adf556c98ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.287103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403f890-81f3-4746-82aa-adf556c98ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.287103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403f890-81f3-4746-82aa-adf556c98ad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.287103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403f890-81f3-4746-82aa-adf556c98ad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.287103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403f890-81f3-4746-82aa-adf556c98ad2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.311596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 628/1239] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/projects => generated 393 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:23.312499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38509610-fb35-44b4-b7c2-49179ea79c47 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.356118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 612/1240] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:23.356118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403f890-81f3-4746-82aa-adf556c98ad2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.359227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:23.374396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38509610-fb35-44b4-b7c2-49179ea79c47 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.405597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:23.420763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403f890-81f3-4746-82aa-adf556c98ad2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.426484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba44b3d-d441-4e6c-bba2-529aca3a5137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.427118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba44b3d-d441-4e6c-bba2-529aca3a5137 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.427401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba44b3d-d441-4e6c-bba2-529aca3a5137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.427669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba44b3d-d441-4e6c-bba2-529aca3a5137 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.428309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba44b3d-d441-4e6c-bba2-529aca3a5137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:23.428682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba44b3d-d441-4e6c-bba2-529aca3a5137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.430273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba44b3d-d441-4e6c-bba2-529aca3a5137 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.433810 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c7273-cd70-4a6a-b761-e419ee41a3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.434430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c7273-cd70-4a6a-b761-e419ee41a3ca admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.434711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c7273-cd70-4a6a-b761-e419ee41a3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.434985 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c7273-cd70-4a6a-b761-e419ee41a3ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.435658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c7273-cd70-4a6a-b761-e419ee41a3ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:23.435961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c7273-cd70-4a6a-b761-e419ee41a3ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.437589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c7273-cd70-4a6a-b761-e419ee41a3ca admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.441999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 613/1241] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:23.441999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba44b3d-d441-4e6c-bba2-529aca3a5137 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.452980 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 629/1242] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:23.452980 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c7273-cd70-4a6a-b761-e419ee41a3ca admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.453825 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba44b3d-d441-4e6c-bba2-529aca3a5137 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.462939 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32dbae-dde3-4c7d-a80f-d21724109082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.463349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c7273-cd70-4a6a-b761-e419ee41a3ca admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.463767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32dbae-dde3-4c7d-a80f-d21724109082 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.464142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32dbae-dde3-4c7d-a80f-d21724109082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.464509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32dbae-dde3-4c7d-a80f-d21724109082 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.465346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32dbae-dde3-4c7d-a80f-d21724109082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:23.465846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32dbae-dde3-4c7d-a80f-d21724109082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.467803 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32dbae-dde3-4c7d-a80f-d21724109082 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.468131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac2828-fb78-46e9-9f23-73e6f5c65e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.468770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac2828-fb78-46e9-9f23-73e6f5c65e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.469057 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac2828-fb78-46e9-9f23-73e6f5c65e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.469652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac2828-fb78-46e9-9f23-73e6f5c65e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.470416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac2828-fb78-46e9-9f23-73e6f5c65e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.472237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac2828-fb78-46e9-9f23-73e6f5c65e76 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.480876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 614/1243] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:23.481795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32dbae-dde3-4c7d-a80f-d21724109082 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.485634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 630/1244] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:23.485634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac2828-fb78-46e9-9f23-73e6f5c65e76 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.492446 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32dbae-dde3-4c7d-a80f-d21724109082 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.496120 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac2828-fb78-46e9-9f23-73e6f5c65e76 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.497276 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7786d6e-be9c-4230-b210-f00a50c082d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.498490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7786d6e-be9c-4230-b210-f00a50c082d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.498944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7786d6e-be9c-4230-b210-f00a50c082d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.499556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7786d6e-be9c-4230-b210-f00a50c082d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.500257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7786d6e-be9c-4230-b210-f00a50c082d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:23.500565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7786d6e-be9c-4230-b210-f00a50c082d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.500675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f069d8-2c86-4078-ab53-59a1a3951a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.501289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f069d8-2c86-4078-ab53-59a1a3951a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.501570 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f069d8-2c86-4078-ab53-59a1a3951a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.501841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f069d8-2c86-4078-ab53-59a1a3951a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.502247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7786d6e-be9c-4230-b210-f00a50c082d9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.503512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f069d8-2c86-4078-ab53-59a1a3951a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:23.503512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f069d8-2c86-4078-ab53-59a1a3951a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.504945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f069d8-2c86-4078-ab53-59a1a3951a22 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.516178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 615/1245] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:23.516301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 631/1246] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:23.517129 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f069d8-2c86-4078-ab53-59a1a3951a22 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.518317 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7786d6e-be9c-4230-b210-f00a50c082d9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.528353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7786d6e-be9c-4230-b210-f00a50c082d9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.532778 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f069d8-2c86-4078-ab53-59a1a3951a22 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.533008 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758edd00-9ba1-4ca0-b78e-c2a624d5797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.533867 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758edd00-9ba1-4ca0-b78e-c2a624d5797d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.534052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758edd00-9ba1-4ca0-b78e-c2a624d5797d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.534327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758edd00-9ba1-4ca0-b78e-c2a624d5797d admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.535130 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758edd00-9ba1-4ca0-b78e-c2a624d5797d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.537053 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758edd00-9ba1-4ca0-b78e-c2a624d5797d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.537592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3708587-0413-4415-ad07-961d582ef4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.538193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3708587-0413-4415-ad07-961d582ef4ed admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.538473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3708587-0413-4415-ad07-961d582ef4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.538745 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3708587-0413-4415-ad07-961d582ef4ed admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.539447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3708587-0413-4415-ad07-961d582ef4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.541335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3708587-0413-4415-ad07-961d582ef4ed admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.642139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 616/1247] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:23.643806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758edd00-9ba1-4ca0-b78e-c2a624d5797d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.654028 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758edd00-9ba1-4ca0-b78e-c2a624d5797d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.658701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4958f8-9b92-4550-aff1-aefaa078085e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.663617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4958f8-9b92-4550-aff1-aefaa078085e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.663617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4958f8-9b92-4550-aff1-aefaa078085e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.663617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4958f8-9b92-4550-aff1-aefaa078085e admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.663617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4958f8-9b92-4550-aff1-aefaa078085e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.663617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4958f8-9b92-4550-aff1-aefaa078085e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.667774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 632/1248] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:23.667774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3708587-0413-4415-ad07-961d582ef4ed admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.676537 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3708587-0413-4415-ad07-961d582ef4ed admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.681615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834421fb-db28-4163-85cc-6a69659129f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.687768 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834421fb-db28-4163-85cc-6a69659129f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.687768 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834421fb-db28-4163-85cc-6a69659129f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.687768 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834421fb-db28-4163-85cc-6a69659129f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.687768 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834421fb-db28-4163-85cc-6a69659129f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.687768 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834421fb-db28-4163-85cc-6a69659129f9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.758849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 617/1249] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:23.759079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4958f8-9b92-4550-aff1-aefaa078085e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.784815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 633/1250] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:23.784815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834421fb-db28-4163-85cc-6a69659129f9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.824588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4958f8-9b92-4550-aff1-aefaa078085e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.844505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834421fb-db28-4163-85cc-6a69659129f9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.849722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2c7c6-ba49-402f-b527-55168a51d086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.850327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2c7c6-ba49-402f-b527-55168a51d086 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.850601 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2c7c6-ba49-402f-b527-55168a51d086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.850872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2c7c6-ba49-402f-b527-55168a51d086 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.851504 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2c7c6-ba49-402f-b527-55168a51d086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.853436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2c7c6-ba49-402f-b527-55168a51d086 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.880665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613efeb-eff0-40fe-a7a9-fbe74aa3b94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.881364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613efeb-eff0-40fe-a7a9-fbe74aa3b94d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.881647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613efeb-eff0-40fe-a7a9-fbe74aa3b94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.881921 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613efeb-eff0-40fe-a7a9-fbe74aa3b94d admin admin] PATH_INFO: `/v3/projects/bc6f0c8072ad445892faa5cb5253357e/users/cfbb59c3a18a4bbaa752cbee63ae128d/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.896119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613efeb-eff0-40fe-a7a9-fbe74aa3b94d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6f0c8072ad445892faa5cb5253357e, user_id=cfbb59c3a18a4bbaa752cbee63ae128d, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.898986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613efeb-eff0-40fe-a7a9-fbe74aa3b94d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:23.929691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 634/1251] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:23.930699 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2c7c6-ba49-402f-b527-55168a51d086 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:23.940099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:23.990499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2c7c6-ba49-402f-b527-55168a51d086 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:23.995336 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf43d8-1225-4729-9c71-5797b37dd8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:23.995940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf43d8-1225-4729-9c71-5797b37dd8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:23.996255 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf43d8-1225-4729-9c71-5797b37dd8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:23.996529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf43d8-1225-4729-9c71-5797b37dd8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:23.997163 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf43d8-1225-4729-9c71-5797b37dd8d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:23.999124 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf43d8-1225-4729-9c71-5797b37dd8d3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.001835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 618/1252] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:23 2022] PUT /identity/v3/projects/bc6f0c8072ad445892faa5cb5253357e/users/cfbb59c3a18a4bbaa752cbee63ae128d/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:24.002705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613efeb-eff0-40fe-a7a9-fbe74aa3b94d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.011343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:24.064577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613efeb-eff0-40fe-a7a9-fbe74aa3b94d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.069578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f71219-4b58-4f3d-a910-fa8d6c244ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.070186 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f71219-4b58-4f3d-a910-fa8d6c244ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.070470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f71219-4b58-4f3d-a910-fa8d6c244ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.070764 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f71219-4b58-4f3d-a910-fa8d6c244ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.071377 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f71219-4b58-4f3d-a910-fa8d6c244ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.073573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f71219-4b58-4f3d-a910-fa8d6c244ae2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.105886 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 635/1253] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:23 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:24.106799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf43d8-1225-4729-9c71-5797b37dd8d3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.167125 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf43d8-1225-4729-9c71-5797b37dd8d3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.171769 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b4820-7a48-4cad-9f6a-913c29bee9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.172377 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b4820-7a48-4cad-9f6a-913c29bee9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.172652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b4820-7a48-4cad-9f6a-913c29bee9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.172923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b4820-7a48-4cad-9f6a-913c29bee9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.173593 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b4820-7a48-4cad-9f6a-913c29bee9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.176365 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b4820-7a48-4cad-9f6a-913c29bee9d7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.181363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 619/1254] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:24 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:24.182233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f71219-4b58-4f3d-a910-fa8d6c244ae2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.192214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 636/1255] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.192214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b4820-7a48-4cad-9f6a-913c29bee9d7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.202527 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b4820-7a48-4cad-9f6a-913c29bee9d7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.246571 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f71219-4b58-4f3d-a910-fa8d6c244ae2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.251277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3f23c-d0ae-4d84-803b-7fa107285cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.251889 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3f23c-d0ae-4d84-803b-7fa107285cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.252169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3f23c-d0ae-4d84-803b-7fa107285cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.252482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3f23c-d0ae-4d84-803b-7fa107285cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.253329 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3f23c-d0ae-4d84-803b-7fa107285cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.254938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3f23c-d0ae-4d84-803b-7fa107285cb8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.258407 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e76983-98c9-440e-8172-ff414af6f0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.259140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e76983-98c9-440e-8172-ff414af6f0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.259427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e76983-98c9-440e-8172-ff414af6f0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.259711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e76983-98c9-440e-8172-ff414af6f0e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.260355 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e76983-98c9-440e-8172-ff414af6f0e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.262396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e76983-98c9-440e-8172-ff414af6f0e9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.264691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 620/1256] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.265490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3f23c-d0ae-4d84-803b-7fa107285cb8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.277506 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3f23c-d0ae-4d84-803b-7fa107285cb8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.282045 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63af119-2f76-4c68-b9c8-067ed6d9362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.282631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63af119-2f76-4c68-b9c8-067ed6d9362c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.282910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63af119-2f76-4c68-b9c8-067ed6d9362c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.283180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63af119-2f76-4c68-b9c8-067ed6d9362c admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.284493 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63af119-2f76-4c68-b9c8-067ed6d9362c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.286368 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63af119-2f76-4c68-b9c8-067ed6d9362c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.362246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 637/1257] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:24 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:24.363070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e76983-98c9-440e-8172-ff414af6f0e9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.372333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e76983-98c9-440e-8172-ff414af6f0e9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.376918 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fee9aa-45da-4410-9741-89a33b499817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.377550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fee9aa-45da-4410-9741-89a33b499817 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.377828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fee9aa-45da-4410-9741-89a33b499817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.383884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fee9aa-45da-4410-9741-89a33b499817 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.383884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fee9aa-45da-4410-9741-89a33b499817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.383884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fee9aa-45da-4410-9741-89a33b499817 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.388252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 621/1258] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:24 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:24.388252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8ddf0-86e6-49ee-b4ea-772f7089f56c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.388252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8ddf0-86e6-49ee-b4ea-772f7089f56c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.388252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8ddf0-86e6-49ee-b4ea-772f7089f56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.406397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-14f8ddf0-86e6-49ee-b4ea-772f7089f56c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:24.407708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-14f8ddf0-86e6-49ee-b4ea-772f7089f56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:24.416041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 622/1259] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:24.416780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8ddf0-86e6-49ee-b4ea-772f7089f56c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.426402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8ddf0-86e6-49ee-b4ea-772f7089f56c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.430949 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c6d19-addd-4b07-95e0-688237857225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.431586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c6d19-addd-4b07-95e0-688237857225 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.431870 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c6d19-addd-4b07-95e0-688237857225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.432146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c6d19-addd-4b07-95e0-688237857225 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.432770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c6d19-addd-4b07-95e0-688237857225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.434897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c6d19-addd-4b07-95e0-688237857225 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.456766 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 638/1260] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:24 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:24.456766 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fee9aa-45da-4410-9741-89a33b499817 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.459866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:24.465032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fee9aa-45da-4410-9741-89a33b499817 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.469709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806fb96-ae2b-41f5-bc38-2e78eccfe3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.470293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806fb96-ae2b-41f5-bc38-2e78eccfe3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.471897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806fb96-ae2b-41f5-bc38-2e78eccfe3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.471897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806fb96-ae2b-41f5-bc38-2e78eccfe3bd admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.472232 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806fb96-ae2b-41f5-bc38-2e78eccfe3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.474183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806fb96-ae2b-41f5-bc38-2e78eccfe3bd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.504781 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 623/1261] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:24 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:24.505825 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c6d19-addd-4b07-95e0-688237857225 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.515604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c6d19-addd-4b07-95e0-688237857225 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.520797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb13d1a-f42c-4f95-a04f-7f8bf3837ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.521404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb13d1a-f42c-4f95-a04f-7f8bf3837ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.521685 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb13d1a-f42c-4f95-a04f-7f8bf3837ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.535152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb13d1a-f42c-4f95-a04f-7f8bf3837ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.535152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb13d1a-f42c-4f95-a04f-7f8bf3837ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.535152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb13d1a-f42c-4f95-a04f-7f8bf3837ef2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.535152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 624/1262] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.535782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb13d1a-f42c-4f95-a04f-7f8bf3837ef2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.547526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb13d1a-f42c-4f95-a04f-7f8bf3837ef2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.552910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811a0e-7e91-4774-841c-ca50161676d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.553531 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811a0e-7e91-4774-841c-ca50161676d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.553813 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811a0e-7e91-4774-841c-ca50161676d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.554087 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811a0e-7e91-4774-841c-ca50161676d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.554758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07811a0e-7e91-4774-841c-ca50161676d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.556364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07811a0e-7e91-4774-841c-ca50161676d8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.566852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 625/1263] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.567816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811a0e-7e91-4774-841c-ca50161676d8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.577509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811a0e-7e91-4774-841c-ca50161676d8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.582997 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396983ee-25f2-4a60-986c-1a84a9c4e2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.588564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396983ee-25f2-4a60-986c-1a84a9c4e2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.588564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396983ee-25f2-4a60-986c-1a84a9c4e2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.588564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396983ee-25f2-4a60-986c-1a84a9c4e2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.588564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396983ee-25f2-4a60-986c-1a84a9c4e2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.588934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 639/1264] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:24 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:24.589029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396983ee-25f2-4a60-986c-1a84a9c4e2f7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.589116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806fb96-ae2b-41f5-bc38-2e78eccfe3bd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.600695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 626/1265] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.600883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806fb96-ae2b-41f5-bc38-2e78eccfe3bd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.600993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396983ee-25f2-4a60-986c-1a84a9c4e2f7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.603046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb102c5-af7a-48bb-b306-254bdb692de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.603652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb102c5-af7a-48bb-b306-254bdb692de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.603931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb102c5-af7a-48bb-b306-254bdb692de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.604204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb102c5-af7a-48bb-b306-254bdb692de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.604847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb102c5-af7a-48bb-b306-254bdb692de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.606513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb102c5-af7a-48bb-b306-254bdb692de6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.615190 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:24.617862 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 640/1266] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.618774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb102c5-af7a-48bb-b306-254bdb692de6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.629339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb102c5-af7a-48bb-b306-254bdb692de6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.633899 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8442deb-6d01-41a8-a97f-f3972bc251b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.634489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8442deb-6d01-41a8-a97f-f3972bc251b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.634769 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8442deb-6d01-41a8-a97f-f3972bc251b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.635039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8442deb-6d01-41a8-a97f-f3972bc251b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.635663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8442deb-6d01-41a8-a97f-f3972bc251b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.637253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8442deb-6d01-41a8-a97f-f3972bc251b7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.647867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 641/1267] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.651891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8442deb-6d01-41a8-a97f-f3972bc251b7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.668312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396983ee-25f2-4a60-986c-1a84a9c4e2f7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.668485 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8442deb-6d01-41a8-a97f-f3972bc251b7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.668603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8638afa-63ab-4e84-a60d-ce6835226f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.668603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8638afa-63ab-4e84-a60d-ce6835226f48 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.668603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8638afa-63ab-4e84-a60d-ce6835226f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.668603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8638afa-63ab-4e84-a60d-ce6835226f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.668937 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586c751-9d11-420d-8628-614376f24e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.668937 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586c751-9d11-420d-8628-614376f24e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.668937 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586c751-9d11-420d-8628-614376f24e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.668937 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586c751-9d11-420d-8628-614376f24e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.669321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586c751-9d11-420d-8628-614376f24e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.669321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586c751-9d11-420d-8628-614376f24e07 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.675368 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8638afa-63ab-4e84-a60d-ce6835226f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.677133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8638afa-63ab-4e84-a60d-ce6835226f48 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.678991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 642/1268] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.679867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586c751-9d11-420d-8628-614376f24e07 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.687523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 627/1269] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:24.688850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8638afa-63ab-4e84-a60d-ce6835226f48 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.689625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586c751-9d11-420d-8628-614376f24e07 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.695424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c65959-d7f7-4edd-9a05-4205066ab1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.696136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c65959-d7f7-4edd-9a05-4205066ab1fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.696565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c65959-d7f7-4edd-9a05-4205066ab1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.696695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c65959-d7f7-4edd-9a05-4205066ab1fc admin admin] PATH_INFO: `/v3/projects/a8129c20cf8d4bc38c4887dcd3d2b968/users/c36b6b9fbf194346bfc0a7e277c72eb4/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.701131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8638afa-63ab-4e84-a60d-ce6835226f48 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.707444 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049d39a-5446-4a35-8741-8864bc3b6504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.708268 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049d39a-5446-4a35-8741-8864bc3b6504 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.708548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049d39a-5446-4a35-8741-8864bc3b6504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.708842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049d39a-5446-4a35-8741-8864bc3b6504 admin admin] PATH_INFO: `/v3/projects/1c598340dc584602a53ca1860d932f87/users/83bf124cd05147718d1d7a062fed7c64/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.713036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c65959-d7f7-4edd-9a05-4205066ab1fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8129c20cf8d4bc38c4887dcd3d2b968, user_id=c36b6b9fbf194346bfc0a7e277c72eb4, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.716165 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c65959-d7f7-4edd-9a05-4205066ab1fc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.725794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049d39a-5446-4a35-8741-8864bc3b6504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c598340dc584602a53ca1860d932f87, user_id=83bf124cd05147718d1d7a062fed7c64, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.728509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049d39a-5446-4a35-8741-8864bc3b6504 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.814212 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 628/1271] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:24 2022] PUT /identity/v3/projects/1c598340dc584602a53ca1860d932f87/users/83bf124cd05147718d1d7a062fed7c64/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:24.814512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 643/1271] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:24 2022] PUT /identity/v3/projects/a8129c20cf8d4bc38c4887dcd3d2b968/users/c36b6b9fbf194346bfc0a7e277c72eb4/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:24.815263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049d39a-5446-4a35-8741-8864bc3b6504 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.816274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c65959-d7f7-4edd-9a05-4205066ab1fc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.895224 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c65959-d7f7-4edd-9a05-4205066ab1fc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.897903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049d39a-5446-4a35-8741-8864bc3b6504 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.900247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4f38f-4ca2-4411-b9e4-40e336a05648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.900864 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4f38f-4ca2-4411-b9e4-40e336a05648 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.901143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4f38f-4ca2-4411-b9e4-40e336a05648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.901458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4f38f-4ca2-4411-b9e4-40e336a05648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.902100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4f38f-4ca2-4411-b9e4-40e336a05648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.902817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f80a80-3f32-41f0-9f4f-951d9fdab848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.903481 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f80a80-3f32-41f0-9f4f-951d9fdab848 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.903759 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f80a80-3f32-41f0-9f4f-951d9fdab848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.903868 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4f38f-4ca2-4411-b9e4-40e336a05648 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.904033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f80a80-3f32-41f0-9f4f-951d9fdab848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.904669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f80a80-3f32-41f0-9f4f-951d9fdab848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.906283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f80a80-3f32-41f0-9f4f-951d9fdab848 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.914301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 644/1272] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.915126 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4f38f-4ca2-4411-b9e4-40e336a05648 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.916436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 629/1273] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.917290 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f80a80-3f32-41f0-9f4f-951d9fdab848 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.925289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4f38f-4ca2-4411-b9e4-40e336a05648 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.930946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00b867-5dee-42f6-90d5-fc809b194e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.931343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f80a80-3f32-41f0-9f4f-951d9fdab848 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.931540 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00b867-5dee-42f6-90d5-fc809b194e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.931975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00b867-5dee-42f6-90d5-fc809b194e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.932183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00b867-5dee-42f6-90d5-fc809b194e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.932845 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00b867-5dee-42f6-90d5-fc809b194e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.934485 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00b867-5dee-42f6-90d5-fc809b194e91 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.936085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ae034-c4ab-4a07-bd16-d9266c3a4c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.936773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ae034-c4ab-4a07-bd16-d9266c3a4c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.937224 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ae034-c4ab-4a07-bd16-d9266c3a4c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.937507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ae034-c4ab-4a07-bd16-d9266c3a4c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.938217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ae034-c4ab-4a07-bd16-d9266c3a4c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.939811 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ae034-c4ab-4a07-bd16-d9266c3a4c85 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.945697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 645/1274] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.946935 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00b867-5dee-42f6-90d5-fc809b194e91 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:24.951328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 630/1275] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.952683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7139837-ce49-42de-95ae-85f28d3a27bd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.953533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7139837-ce49-42de-95ae-85f28d3a27bd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.953825 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7139837-ce49-42de-95ae-85f28d3a27bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.957892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00b867-5dee-42f6-90d5-fc809b194e91 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:24.962983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a76639-e874-4dfb-b13a-3cdf20c966ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:24.966423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a76639-e874-4dfb-b13a-3cdf20c966ce admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.966423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a76639-e874-4dfb-b13a-3cdf20c966ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.966423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a76639-e874-4dfb-b13a-3cdf20c966ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.966423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a76639-e874-4dfb-b13a-3cdf20c966ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:24.966796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a76639-e874-4dfb-b13a-3cdf20c966ce admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:24.976736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a7139837-ce49-42de-95ae-85f28d3a27bd None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:24.977538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 646/1276] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:24.978002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a7139837-ce49-42de-95ae-85f28d3a27bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:24.978778 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe196993-a5e9-4b31-8dfb-a0a0c65babcd None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:24.979049 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe196993-a5e9-4b31-8dfb-a0a0c65babcd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:24.979329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe196993-a5e9-4b31-8dfb-a0a0c65babcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:24.986070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 631/1277] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:24.990499 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7139837-ce49-42de-95ae-85f28d3a27bd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.002584 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fe196993-a5e9-4b31-8dfb-a0a0c65babcd None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:25.003954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fe196993-a5e9-4b31-8dfb-a0a0c65babcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:25.016184 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7139837-ce49-42de-95ae-85f28d3a27bd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.017296 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 647/1278] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:25.017296 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe196993-a5e9-4b31-8dfb-a0a0c65babcd None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.018678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59dc76-2161-43b9-90aa-00cec95afb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.018678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59dc76-2161-43b9-90aa-00cec95afb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.018678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59dc76-2161-43b9-90aa-00cec95afb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.018678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59dc76-2161-43b9-90aa-00cec95afb27 admin admin] PATH_INFO: `/v3/projects/8a33b99412544033a72f5aff6928917d/users/52fbbabc3d144a79accb83c78e3f9e52/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.023625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe196993-a5e9-4b31-8dfb-a0a0c65babcd None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.028750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf9614-eafd-41b2-b63d-e49387924adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.029403 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf9614-eafd-41b2-b63d-e49387924adc admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.029688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf9614-eafd-41b2-b63d-e49387924adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.029966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf9614-eafd-41b2-b63d-e49387924adc admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.030621 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf9614-eafd-41b2-b63d-e49387924adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.032259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf9614-eafd-41b2-b63d-e49387924adc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.037623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59dc76-2161-43b9-90aa-00cec95afb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a33b99412544033a72f5aff6928917d, user_id=52fbbabc3d144a79accb83c78e3f9e52, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.040490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59dc76-2161-43b9-90aa-00cec95afb27 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.044334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 648/1279] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:25.046186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf9614-eafd-41b2-b63d-e49387924adc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.056876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf9614-eafd-41b2-b63d-e49387924adc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.061857 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af046f-5052-410d-9c08-ebd64a798999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.062508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af046f-5052-410d-9c08-ebd64a798999 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.062906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af046f-5052-410d-9c08-ebd64a798999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.063181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af046f-5052-410d-9c08-ebd64a798999 admin admin] PATH_INFO: `/v3/projects/71b9d960cf934d7483bb902bf3315922/users/7a377d8d72e444b2951259c16404a706/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.081449 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af046f-5052-410d-9c08-ebd64a798999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b9d960cf934d7483bb902bf3315922, user_id=7a377d8d72e444b2951259c16404a706, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.083989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af046f-5052-410d-9c08-ebd64a798999 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.128044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 632/1280] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:24 2022] PUT /identity/v3/projects/8a33b99412544033a72f5aff6928917d/users/52fbbabc3d144a79accb83c78e3f9e52/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:25.128563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59dc76-2161-43b9-90aa-00cec95afb27 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.165330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 649/1281] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:25 2022] PUT /identity/v3/projects/71b9d960cf934d7483bb902bf3315922/users/7a377d8d72e444b2951259c16404a706/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:25.165330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af046f-5052-410d-9c08-ebd64a798999 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.179426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:25.181937 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:25.193961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59dc76-2161-43b9-90aa-00cec95afb27 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.236609 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af046f-5052-410d-9c08-ebd64a798999 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.241961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d0f13-7724-4bd3-8f02-d5721f13567d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.242627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d0f13-7724-4bd3-8f02-d5721f13567d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.242938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d0f13-7724-4bd3-8f02-d5721f13567d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.243223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d0f13-7724-4bd3-8f02-d5721f13567d admin admin] PATH_INFO: `/v3/projects/0e99f2fb40134281916636d4c2ef69fa/users/51adf5d219a74f26a1e70cdf65c771f6/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.259511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee315bf-36f1-480a-a777-49910d1655c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.260187 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee315bf-36f1-480a-a777-49910d1655c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.260472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee315bf-36f1-480a-a777-49910d1655c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.260750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee315bf-36f1-480a-a777-49910d1655c6 admin admin] PATH_INFO: `/v3/projects/6df0ffb308eb4b078463b57e47dca40c/users/f2b548d559b54575afd98739340ccf8e/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.263385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d0f13-7724-4bd3-8f02-d5721f13567d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e99f2fb40134281916636d4c2ef69fa, user_id=51adf5d219a74f26a1e70cdf65c771f6, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.266201 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d0f13-7724-4bd3-8f02-d5721f13567d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.289155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee315bf-36f1-480a-a777-49910d1655c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df0ffb308eb4b078463b57e47dca40c, user_id=f2b548d559b54575afd98739340ccf8e, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.289155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee315bf-36f1-480a-a777-49910d1655c6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.339795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 650/1282] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:25 2022] PUT /identity/v3/projects/0e99f2fb40134281916636d4c2ef69fa/users/51adf5d219a74f26a1e70cdf65c771f6/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:25.340802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d0f13-7724-4bd3-8f02-d5721f13567d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.342471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-785d0f13-7724-4bd3-8f02-d5721f13567d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:25.359144 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 633/1283] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:25 2022] PUT /identity/v3/projects/6df0ffb308eb4b078463b57e47dca40c/users/f2b548d559b54575afd98739340ccf8e/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:25.360171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee315bf-36f1-480a-a777-49910d1655c6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.361789 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6ee315bf-36f1-480a-a777-49910d1655c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:25.411453 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d0f13-7724-4bd3-8f02-d5721f13567d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.422479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee315bf-36f1-480a-a777-49910d1655c6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.426670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88faa07-a944-4a84-a4e0-b7c8b25d2787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.427463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88faa07-a944-4a84-a4e0-b7c8b25d2787 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.427806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88faa07-a944-4a84-a4e0-b7c8b25d2787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.428168 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88faa07-a944-4a84-a4e0-b7c8b25d2787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.470588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfe193-b195-4ecf-a0d2-6145e5c3c1bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.471313 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfe193-b195-4ecf-a0d2-6145e5c3c1bf service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.471672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfe193-b195-4ecf-a0d2-6145e5c3c1bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.472272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfe193-b195-4ecf-a0d2-6145e5c3c1bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.496593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88faa07-a944-4a84-a4e0-b7c8b25d2787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.498234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88faa07-a944-4a84-a4e0-b7c8b25d2787 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.510390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 634/1284] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:25.511411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88faa07-a944-4a84-a4e0-b7c8b25d2787 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.521925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88faa07-a944-4a84-a4e0-b7c8b25d2787 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.527257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c920c-508b-4b1c-a7fc-aea2133dce04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.527937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c920c-508b-4b1c-a7fc-aea2133dce04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.528213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c920c-508b-4b1c-a7fc-aea2133dce04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.528477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c920c-508b-4b1c-a7fc-aea2133dce04 admin admin] PATH_INFO: `/v3/projects/f161707b40e04d3089ca050e79cb52bf/users/08d4945f6db94a099e28fd7eadc4fa79/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.545653 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfe193-b195-4ecf-a0d2-6145e5c3c1bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.546821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c920c-508b-4b1c-a7fc-aea2133dce04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f161707b40e04d3089ca050e79cb52bf, user_id=08d4945f6db94a099e28fd7eadc4fa79, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.547556 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfe193-b195-4ecf-a0d2-6145e5c3c1bf service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.549776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c920c-508b-4b1c-a7fc-aea2133dce04 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.561072 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 651/1285] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:25.562401 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8fb1e-89bb-4a86-881a-8f9cff0fd974 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.562691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8fb1e-89bb-4a86-881a-8f9cff0fd974 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.562969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8fb1e-89bb-4a86-881a-8f9cff0fd974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.583831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:25.592534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-04c8fb1e-89bb-4a86-881a-8f9cff0fd974 None None] MFA Rules not processed for user `52fbbabc3d144a79accb83c78e3f9e52`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:25.593814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-04c8fb1e-89bb-4a86-881a-8f9cff0fd974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:25.625365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 635/1286] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:25 2022] PUT /identity/v3/projects/f161707b40e04d3089ca050e79cb52bf/users/08d4945f6db94a099e28fd7eadc4fa79/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:25.626990 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1eeb8-5d12-48f0-8309-c9539b180d51 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.627538 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1eeb8-5d12-48f0-8309-c9539b180d51 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.627823 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1eeb8-5d12-48f0-8309-c9539b180d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.635056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:25.656712 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-aaf1eeb8-5d12-48f0-8309-c9539b180d51 None None] MFA Rules not processed for user `7a377d8d72e444b2951259c16404a706`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:25.657991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aaf1eeb8-5d12-48f0-8309-c9539b180d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:25.663218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 652/1287] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:25.664685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4d887-0c57-4594-8bb3-76fb8f652a17 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.664977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4d887-0c57-4594-8bb3-76fb8f652a17 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.665294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4d887-0c57-4594-8bb3-76fb8f652a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.686540 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-67a4d887-0c57-4594-8bb3-76fb8f652a17 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:25.693462 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-67a4d887-0c57-4594-8bb3-76fb8f652a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:25.749979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 636/1288] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:25.755065 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcf6e3-388d-4b8f-98c2-05b5edb74c6e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.755362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcf6e3-388d-4b8f-98c2-05b5edb74c6e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.755641 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcf6e3-388d-4b8f-98c2-05b5edb74c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.756333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 653/1289] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:25.757625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3bb60-1919-4b7b-9629-9b26ff8f88f1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.757969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3bb60-1919-4b7b-9629-9b26ff8f88f1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.758255 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3bb60-1919-4b7b-9629-9b26ff8f88f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.782938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7cfcf6e3-388d-4b8f-98c2-05b5edb74c6e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:25.784911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7cfcf6e3-388d-4b8f-98c2-05b5edb74c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:25.790803 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-66f3bb60-1919-4b7b-9629-9b26ff8f88f1 None None] MFA Rules not processed for user `08d4945f6db94a099e28fd7eadc4fa79`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:25.793301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-66f3bb60-1919-4b7b-9629-9b26ff8f88f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:25.796969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 637/1290] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:25.796969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad9c78-af50-462b-9af2-d2d2461df2ec None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.796969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad9c78-af50-462b-9af2-d2d2461df2ec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.796969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad9c78-af50-462b-9af2-d2d2461df2ec None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.797343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 638/1291] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 13:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:03:25.797908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad9c78-af50-462b-9af2-d2d2461df2ec None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.812407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad9c78-af50-462b-9af2-d2d2461df2ec None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.818755 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae41d0-0897-4204-bce5-f1f6928b9362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.822019 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae41d0-0897-4204-bce5-f1f6928b9362 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.822019 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae41d0-0897-4204-bce5-f1f6928b9362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.822019 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae41d0-0897-4204-bce5-f1f6928b9362 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.822019 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae41d0-0897-4204-bce5-f1f6928b9362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.822588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae41d0-0897-4204-bce5-f1f6928b9362 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.878282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 654/1292] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:25.879251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3bb60-1919-4b7b-9629-9b26ff8f88f1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:25.909675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:25.935612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3bb60-1919-4b7b-9629-9b26ff8f88f1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:25.939405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21c47b-d6b6-40c3-93bc-5f3e02d9d591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:25.940648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21c47b-d6b6-40c3-93bc-5f3e02d9d591 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.940927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21c47b-d6b6-40c3-93bc-5f3e02d9d591 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.941421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21c47b-d6b6-40c3-93bc-5f3e02d9d591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.950988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21c47b-d6b6-40c3-93bc-5f3e02d9d591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:25.952451 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21c47b-d6b6-40c3-93bc-5f3e02d9d591 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:25.962665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 655/1293] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:25.963932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429d5b7-8b26-4e64-8212-9a67faf1d4fa None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:25.964467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429d5b7-8b26-4e64-8212-9a67faf1d4fa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:25.964752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429d5b7-8b26-4e64-8212-9a67faf1d4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:25.984449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 639/1294] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:25 2022] POST /identity/v3/projects => generated 363 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:25.986299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae41d0-0897-4204-bce5-f1f6928b9362 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.013783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c429d5b7-8b26-4e64-8212-9a67faf1d4fa None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:26.014994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c429d5b7-8b26-4e64-8212-9a67faf1d4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:26.052741 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae41d0-0897-4204-bce5-f1f6928b9362 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.056421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d3cb4-54d2-46f0-8d06-e3ee4dc2e3a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.057398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d3cb4-54d2-46f0-8d06-e3ee4dc2e3a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.057398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d3cb4-54d2-46f0-8d06-e3ee4dc2e3a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.057687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d3cb4-54d2-46f0-8d06-e3ee4dc2e3a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.099917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 656/1295] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 13:03:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:26.110825 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429d5b7-8b26-4e64-8212-9a67faf1d4fa None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.132474 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d3cb4-54d2-46f0-8d06-e3ee4dc2e3a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.134669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d3cb4-54d2-46f0-8d06-e3ee4dc2e3a7 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.147423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 640/1296] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:26.148590 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d3cb4-54d2-46f0-8d06-e3ee4dc2e3a7 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.160281 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d3cb4-54d2-46f0-8d06-e3ee4dc2e3a7 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.167069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3249c-891a-4f75-82e0-03b7616f2c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.167795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3249c-891a-4f75-82e0-03b7616f2c9e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.168079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3249c-891a-4f75-82e0-03b7616f2c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.168352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3249c-891a-4f75-82e0-03b7616f2c9e admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.169214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3249c-891a-4f75-82e0-03b7616f2c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.172579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3249c-891a-4f75-82e0-03b7616f2c9e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.190907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:26.195461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429d5b7-8b26-4e64-8212-9a67faf1d4fa None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.200652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14c950-b48f-46bb-94d9-d713762e8790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.201437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14c950-b48f-46bb-94d9-d713762e8790 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.201798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14c950-b48f-46bb-94d9-d713762e8790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.202140 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14c950-b48f-46bb-94d9-d713762e8790 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.202857 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14c950-b48f-46bb-94d9-d713762e8790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.204955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14c950-b48f-46bb-94d9-d713762e8790 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.242911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 641/1297] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:26 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:26.243827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3249c-891a-4f75-82e0-03b7616f2c9e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.264926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3249c-891a-4f75-82e0-03b7616f2c9e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.273267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34513f4f-5e0b-4a43-98ea-6abcbc37fb2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.274076 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34513f4f-5e0b-4a43-98ea-6abcbc37fb2d service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.274417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34513f4f-5e0b-4a43-98ea-6abcbc37fb2d service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.274840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34513f4f-5e0b-4a43-98ea-6abcbc37fb2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.318266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 657/1298] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:26 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:26.319479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14c950-b48f-46bb-94d9-d713762e8790 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.388295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:26.389334 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34513f4f-5e0b-4a43-98ea-6abcbc37fb2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.391565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34513f4f-5e0b-4a43-98ea-6abcbc37fb2d service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.423384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14c950-b48f-46bb-94d9-d713762e8790 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.423606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 642/1299] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:26.424775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34513f4f-5e0b-4a43-98ea-6abcbc37fb2d service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.431170 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b065fd3-3f0e-4c81-a3d0-c3bf12d92ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.434376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b065fd3-3f0e-4c81-a3d0-c3bf12d92ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.434376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b065fd3-3f0e-4c81-a3d0-c3bf12d92ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.434376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b065fd3-3f0e-4c81-a3d0-c3bf12d92ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.434376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b065fd3-3f0e-4c81-a3d0-c3bf12d92ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.437287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b065fd3-3f0e-4c81-a3d0-c3bf12d92ab5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.452390 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 658/1300] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:26.452584 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34513f4f-5e0b-4a43-98ea-6abcbc37fb2d service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.456350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b065fd3-3f0e-4c81-a3d0-c3bf12d92ab5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.465296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aca510-a77a-4ca1-969b-f5561d60f1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.466010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aca510-a77a-4ca1-969b-f5561d60f1fc admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.466353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aca510-a77a-4ca1-969b-f5561d60f1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.466769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aca510-a77a-4ca1-969b-f5561d60f1fc admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.467546 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aca510-a77a-4ca1-969b-f5561d60f1fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.469785 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aca510-a77a-4ca1-969b-f5561d60f1fc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.479136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b065fd3-3f0e-4c81-a3d0-c3bf12d92ab5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.493229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec42b3-9bbe-436a-964f-9e7967c50751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.493430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec42b3-9bbe-436a-964f-9e7967c50751 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.495318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec42b3-9bbe-436a-964f-9e7967c50751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.496274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec42b3-9bbe-436a-964f-9e7967c50751 admin admin] PATH_INFO: `/v3/projects/f3c7052b73664cdf93b0bd738ce2b5f7/users/19c19255331643d1ab51a6dd8aa009df/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.529944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec42b3-9bbe-436a-964f-9e7967c50751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c7052b73664cdf93b0bd738ce2b5f7, user_id=19c19255331643d1ab51a6dd8aa009df, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.534610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec42b3-9bbe-436a-964f-9e7967c50751 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.656687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 643/1302] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:26 2022] POST /identity/v3/users => generated 299 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:26.660325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 659/1302] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:26 2022] PUT /identity/v3/projects/f3c7052b73664cdf93b0bd738ce2b5f7/users/19c19255331643d1ab51a6dd8aa009df/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:03:26.660477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aca510-a77a-4ca1-969b-f5561d60f1fc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.661482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec42b3-9bbe-436a-964f-9e7967c50751 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.684350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aca510-a77a-4ca1-969b-f5561d60f1fc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.796720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae0366-b0fb-4cdf-9d8e-8d46b8f4fd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.797689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec42b3-9bbe-436a-964f-9e7967c50751 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.798746 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae0366-b0fb-4cdf-9d8e-8d46b8f4fd1f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.799035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae0366-b0fb-4cdf-9d8e-8d46b8f4fd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.799886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae0366-b0fb-4cdf-9d8e-8d46b8f4fd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.801796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae0366-b0fb-4cdf-9d8e-8d46b8f4fd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.804510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae0366-b0fb-4cdf-9d8e-8d46b8f4fd1f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.809566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9baec-7569-491d-a4ba-0a5897896322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.810271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9baec-7569-491d-a4ba-0a5897896322 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.810557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9baec-7569-491d-a4ba-0a5897896322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.810830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9baec-7569-491d-a4ba-0a5897896322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.811484 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9baec-7569-491d-a4ba-0a5897896322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.818531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9baec-7569-491d-a4ba-0a5897896322 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.833089 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 644/1303] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:26.834108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae0366-b0fb-4cdf-9d8e-8d46b8f4fd1f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.845243 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 660/1304] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:26.853058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9baec-7569-491d-a4ba-0a5897896322 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:26.859013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae0366-b0fb-4cdf-9d8e-8d46b8f4fd1f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.867616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9baec-7569-491d-a4ba-0a5897896322 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:26.871443 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34459157-7fb3-4fcd-97b2-73ddc966f7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.872138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34459157-7fb3-4fcd-97b2-73ddc966f7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.872485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34459157-7fb3-4fcd-97b2-73ddc966f7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.873039 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34459157-7fb3-4fcd-97b2-73ddc966f7b4 admin admin] PATH_INFO: `/v3/projects/f3c7052b73664cdf93b0bd738ce2b5f7/users/19c19255331643d1ab51a6dd8aa009df/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.877161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34459157-7fb3-4fcd-97b2-73ddc966f7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c7052b73664cdf93b0bd738ce2b5f7, user_id=19c19255331643d1ab51a6dd8aa009df, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.879856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f8178-090a-469c-88b0-19df7076d0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:26.881975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f8178-090a-469c-88b0-19df7076d0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.882273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f8178-090a-469c-88b0-19df7076d0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.882554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f8178-090a-469c-88b0-19df7076d0c7 admin admin] PATH_INFO: `/v3/projects/b16a2a5eb92c4940b20d21871d6abe4b/users/be777ec5c77a4f3a8ddb4930f9bcdf35/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.884013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34459157-7fb3-4fcd-97b2-73ddc966f7b4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.910974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 645/1305] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:26 2022] PUT /identity/v3/projects/f3c7052b73664cdf93b0bd738ce2b5f7/users/19c19255331643d1ab51a6dd8aa009df/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:26.922723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140fc22-f69e-41e1-9d21-4e67b1c9646a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:26.923186 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140fc22-f69e-41e1-9d21-4e67b1c9646a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:26.923309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140fc22-f69e-41e1-9d21-4e67b1c9646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:26.928783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f8178-090a-469c-88b0-19df7076d0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16a2a5eb92c4940b20d21871d6abe4b, user_id=be777ec5c77a4f3a8ddb4930f9bcdf35, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:26.931950 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f8178-090a-469c-88b0-19df7076d0c7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:26.967286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3140fc22-f69e-41e1-9d21-4e67b1c9646a None None] MFA Rules not processed for user `19c19255331643d1ab51a6dd8aa009df`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:26.968820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3140fc22-f69e-41e1-9d21-4e67b1c9646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:27.030357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 661/1306] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:26 2022] PUT /identity/v3/projects/b16a2a5eb92c4940b20d21871d6abe4b/users/be777ec5c77a4f3a8ddb4930f9bcdf35/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:27.031618 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f8178-090a-469c-88b0-19df7076d0c7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:27.043265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:27.100605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 646/1307] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:27.106079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140fc22-f69e-41e1-9d21-4e67b1c9646a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:27.202553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f8178-090a-469c-88b0-19df7076d0c7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:27.222139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a48513-c619-4fdf-a418-39cdafaf6fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:27.222445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a48513-c619-4fdf-a418-39cdafaf6fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.222775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a48513-c619-4fdf-a418-39cdafaf6fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.223741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a48513-c619-4fdf-a418-39cdafaf6fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.223741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a48513-c619-4fdf-a418-39cdafaf6fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:27.229646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a48513-c619-4fdf-a418-39cdafaf6fb9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:27.234913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140fc22-f69e-41e1-9d21-4e67b1c9646a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:27.259673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 662/1308] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:27.259673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a48513-c619-4fdf-a418-39cdafaf6fb9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:27.282213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a48513-c619-4fdf-a418-39cdafaf6fb9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:27.289321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb961a92-8221-46ef-86bd-2a79394e9bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:27.289535 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb961a92-8221-46ef-86bd-2a79394e9bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.289915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb961a92-8221-46ef-86bd-2a79394e9bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.290284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb961a92-8221-46ef-86bd-2a79394e9bc2 admin admin] PATH_INFO: `/v3/projects/b16a2a5eb92c4940b20d21871d6abe4b/users/be777ec5c77a4f3a8ddb4930f9bcdf35/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.293142 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb961a92-8221-46ef-86bd-2a79394e9bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16a2a5eb92c4940b20d21871d6abe4b, user_id=be777ec5c77a4f3a8ddb4930f9bcdf35, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:27.296473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb961a92-8221-46ef-86bd-2a79394e9bc2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:27.323248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 663/1309] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:27 2022] PUT /identity/v3/projects/b16a2a5eb92c4940b20d21871d6abe4b/users/be777ec5c77a4f3a8ddb4930f9bcdf35/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:27.327854 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450723c-011c-4543-bf87-a9ba054abdda None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.328376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450723c-011c-4543-bf87-a9ba054abdda None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.328825 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450723c-011c-4543-bf87-a9ba054abdda None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.332055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 664/1310] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 13:03:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:03:27.338196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f80665-fdc0-46f4-9829-b8bbe2a0d73d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.340114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f80665-fdc0-46f4-9829-b8bbe2a0d73d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.340114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f80665-fdc0-46f4-9829-b8bbe2a0d73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.342344 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccf7a5-17d6-4be1-b719-f6b4f00ea6dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:27.343253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccf7a5-17d6-4be1-b719-f6b4f00ea6dc service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.343738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccf7a5-17d6-4be1-b719-f6b4f00ea6dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.344202 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccf7a5-17d6-4be1-b719-f6b4f00ea6dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.358366 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:27.407421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-72f80665-fdc0-46f4-9829-b8bbe2a0d73d None None] MFA Rules not processed for user `be777ec5c77a4f3a8ddb4930f9bcdf35`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:27.409388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-72f80665-fdc0-46f4-9829-b8bbe2a0d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:27.437549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccf7a5-17d6-4be1-b719-f6b4f00ea6dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:27.440238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccf7a5-17d6-4be1-b719-f6b4f00ea6dc service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:27.456922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 647/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:27.458467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baaab0-b6e6-4eb9-8f8f-73c80b0a1e72 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.458697 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baaab0-b6e6-4eb9-8f8f-73c80b0a1e72 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.458977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baaab0-b6e6-4eb9-8f8f-73c80b0a1e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.486030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-14baaab0-b6e6-4eb9-8f8f-73c80b0a1e72 None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:27.487428 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-14baaab0-b6e6-4eb9-8f8f-73c80b0a1e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:27.520220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 665/1312] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:27.532960 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f80665-fdc0-46f4-9829-b8bbe2a0d73d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:27.609631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f80665-fdc0-46f4-9829-b8bbe2a0d73d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:27.614221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765abb58-49b7-4b85-aa34-8a0c1685a37d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:27.614943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765abb58-49b7-4b85-aa34-8a0c1685a37d service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.615318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765abb58-49b7-4b85-aa34-8a0c1685a37d service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.615663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765abb58-49b7-4b85-aa34-8a0c1685a37d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.628403 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 648/1313] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 13:03:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:27.629107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba0368-37fc-4654-b3bc-93b111538611 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.633122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba0368-37fc-4654-b3bc-93b111538611 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.633420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba0368-37fc-4654-b3bc-93b111538611 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.635181 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 649/1314] 10.222.0.30 () {54 vars in 912 bytes} [Sat Oct 15 13:03:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:03:27.637446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765abb58-49b7-4b85-aa34-8a0c1685a37d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:27.639271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765abb58-49b7-4b85-aa34-8a0c1685a37d service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:27.644451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc834c-d265-472f-9cec-c5311fbe8df2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.644828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc834c-d265-472f-9cec-c5311fbe8df2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.645111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc834c-d265-472f-9cec-c5311fbe8df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.676518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 666/1315] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:27.706860 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-31fc834c-d265-472f-9cec-c5311fbe8df2 None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:27.709949 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-31fc834c-d265-472f-9cec-c5311fbe8df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:27.715036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765abb58-49b7-4b85-aa34-8a0c1685a37d service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:27.740428 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 650/1316] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 13:03:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:27.764551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc834c-d265-472f-9cec-c5311fbe8df2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:27.777365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc834c-d265-472f-9cec-c5311fbe8df2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:27.785765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b465a-13d7-4a10-8313-92104d1d3e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:27.787512 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b465a-13d7-4a10-8313-92104d1d3e80 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.789175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b465a-13d7-4a10-8313-92104d1d3e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.789743 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b465a-13d7-4a10-8313-92104d1d3e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.821054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b465a-13d7-4a10-8313-92104d1d3e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:27.825098 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b465a-13d7-4a10-8313-92104d1d3e80 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:27.837616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765abb58-49b7-4b85-aa34-8a0c1685a37d service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:27.843513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:27.844520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:27.844915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:27.845425 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] PATH_INFO: `/v3/users/08d4945f6db94a099e28fd7eadc4fa79` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:27.848203 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d4945f6db94a099e28fd7eadc4fa79)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:27.849440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:03:27.849440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 15 13:03:27.849440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: Oct 15 13:03:27.850944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:27.860448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 651/1317] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:28.159262 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155944cb-5338-4656-a65e-ae8358932c34 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:28.159588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155944cb-5338-4656-a65e-ae8358932c34 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:28.165177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155944cb-5338-4656-a65e-ae8358932c34 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:28.165177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 652/1318] 10.222.0.30 () {54 vars in 912 bytes} [Sat Oct 15 13:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:03:28.168274 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3efe4d-1b41-4b78-bbac-ef025bbfc8ee None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:28.168563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3efe4d-1b41-4b78-bbac-ef025bbfc8ee None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:28.168926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3efe4d-1b41-4b78-bbac-ef025bbfc8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:28.201369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d4945f6db94a099e28fd7eadc4fa79'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:28.216773 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d4945f6db94a099e28fd7eadc4fa79'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:28.217818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d4945f6db94a099e28fd7eadc4fa79'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:28.218760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d4945f6db94a099e28fd7eadc4fa79'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:28.245345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2e3efe4d-1b41-4b78-bbac-ef025bbfc8ee None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:28.246706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2e3efe4d-1b41-4b78-bbac-ef025bbfc8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:28.269408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 653/1319] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 13:03:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:03:28.283167 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3efe4d-1b41-4b78-bbac-ef025bbfc8ee None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:28.291912 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2e3efe4d-1b41-4b78-bbac-ef025bbfc8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:28.298783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3efe4d-1b41-4b78-bbac-ef025bbfc8ee None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:28.308599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9599e2a-b40e-4d0e-8bf0-7c1e23fa17b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:28.309422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9599e2a-b40e-4d0e-8bf0-7c1e23fa17b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:28.309817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9599e2a-b40e-4d0e-8bf0-7c1e23fa17b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:28.310206 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9599e2a-b40e-4d0e-8bf0-7c1e23fa17b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:28.311584 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d4945f6db94a099e28fd7eadc4fa79'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:28.330246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9599e2a-b40e-4d0e-8bf0-7c1e23fa17b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:28.333452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9599e2a-b40e-4d0e-8bf0-7c1e23fa17b6 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:28.357130 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 654/1320] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:03:28.391217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:28.413889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 667/1321] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:03:27 2022] DELETE /identity/v3/users/08d4945f6db94a099e28fd7eadc4fa79 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:03:28.420176 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03101926-b4e2-4383-abd9-9debe90caffa None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:28.420176 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03101926-b4e2-4383-abd9-9debe90caffa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:28.420176 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03101926-b4e2-4383-abd9-9debe90caffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:28.443686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-03101926-b4e2-4383-abd9-9debe90caffa None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:28.445034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-03101926-b4e2-4383-abd9-9debe90caffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:28.511197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:28.513496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:28.637742 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 655/1322] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:28.681783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03101926-b4e2-4383-abd9-9debe90caffa None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:28.684355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-03101926-b4e2-4383-abd9-9debe90caffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:28.792528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07224-6f8e-400a-978e-004a42f26501 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:28.802627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:28.803487 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:28.803762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:28.804031 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] PATH_INFO: `/v3/users/52fbbabc3d144a79accb83c78e3f9e52` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:28.805526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=52fbbabc3d144a79accb83c78e3f9e52)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:28.808988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:28.818426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03101926-b4e2-4383-abd9-9debe90caffa None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:28.822515 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dd32e-d624-4b29-bd7b-7fab41228aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:28.823348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dd32e-d624-4b29-bd7b-7fab41228aab service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:28.823532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dd32e-d624-4b29-bd7b-7fab41228aab service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:28.823811 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dd32e-d624-4b29-bd7b-7fab41228aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:28.839494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dd32e-d624-4b29-bd7b-7fab41228aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:28.841748 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dd32e-d624-4b29-bd7b-7fab41228aab service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:28.874352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 656/1323] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:28.995967 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:29.001502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dd32e-d624-4b29-bd7b-7fab41228aab service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:29.003296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-048dd32e-d624-4b29-bd7b-7fab41228aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:29.023712 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dd32e-d624-4b29-bd7b-7fab41228aab service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:29.040640 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:29.041353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:29.041637 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:29.041929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] PATH_INFO: `/v3/users/7a377d8d72e444b2951259c16404a706` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:29.043627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a377d8d72e444b2951259c16404a706)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:29.045861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:29.134654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52fbbabc3d144a79accb83c78e3f9e52'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.159876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52fbbabc3d144a79accb83c78e3f9e52'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.162008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52fbbabc3d144a79accb83c78e3f9e52'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.164185 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52fbbabc3d144a79accb83c78e3f9e52'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.301502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63a4fb12-2472-467a-b267-7200c0e2cf3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52fbbabc3d144a79accb83c78e3f9e52'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.339956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 668/1324] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:03:28 2022] DELETE /identity/v3/users/52fbbabc3d144a79accb83c78e3f9e52 => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:29.345143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461e6c-f16e-485c-809e-6f5dc4a64276 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:29.345760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461e6c-f16e-485c-809e-6f5dc4a64276 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:29.346548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461e6c-f16e-485c-809e-6f5dc4a64276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:29.373492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:29.391330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a377d8d72e444b2951259c16404a706'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.396452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a377d8d72e444b2951259c16404a706'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.415915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a377d8d72e444b2951259c16404a706'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.417598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a377d8d72e444b2951259c16404a706'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.424498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-26461e6c-f16e-485c-809e-6f5dc4a64276 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:29.426158 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-26461e6c-f16e-485c-809e-6f5dc4a64276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:29.527640 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a377d8d72e444b2951259c16404a706'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:29.570632 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 669/1325] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:29.579640 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 657/1326] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:03:28 2022] DELETE /identity/v3/users/7a377d8d72e444b2951259c16404a706 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:03:29.584987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf64c4-02e5-4390-9e09-8e0ba630c664 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:29.585513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf64c4-02e5-4390-9e09-8e0ba630c664 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:29.589930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf64c4-02e5-4390-9e09-8e0ba630c664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:29.598596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:29.600647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:29.632023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b3bf64c4-02e5-4390-9e09-8e0ba630c664 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:29.636613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b3bf64c4-02e5-4390-9e09-8e0ba630c664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:29.758751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e24a36-88a8-4e7a-ba76-2b5f0190a88c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:29.772666 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41309c0c-0d44-4683-90fa-5dc0e40d76c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:29.774690 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41309c0c-0d44-4683-90fa-5dc0e40d76c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:29.775975 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41309c0c-0d44-4683-90fa-5dc0e40d76c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:29.776258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41309c0c-0d44-4683-90fa-5dc0e40d76c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:29.806046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 670/1327] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:03:29.829973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf64c4-02e5-4390-9e09-8e0ba630c664 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:29.861177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf64c4-02e5-4390-9e09-8e0ba630c664 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:29.870889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c07b7-d9b4-4011-803a-e0aae83ed329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:29.880103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c07b7-d9b4-4011-803a-e0aae83ed329 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:29.880103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c07b7-d9b4-4011-803a-e0aae83ed329 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:29.880103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c07b7-d9b4-4011-803a-e0aae83ed329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:29.898151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c07b7-d9b4-4011-803a-e0aae83ed329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:29.904031 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c07b7-d9b4-4011-803a-e0aae83ed329 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:29.937372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 671/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:29.937547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41309c0c-0d44-4683-90fa-5dc0e40d76c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:29.938876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41309c0c-0d44-4683-90fa-5dc0e40d76c9 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:29.954832 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 658/1329] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:30.007866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c07b7-d9b4-4011-803a-e0aae83ed329 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:30.009583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bf6c07b7-d9b4-4011-803a-e0aae83ed329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:30.025510 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c07b7-d9b4-4011-803a-e0aae83ed329 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:30.032432 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:30.038298 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:30.038969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:30.039533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] PATH_INFO: `/v3/projects/f161707b40e04d3089ca050e79cb52bf` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:30.041312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f161707b40e04d3089ca050e79cb52bf)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:30.043948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:30.116607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:30.232342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f161707b40e04d3089ca050e79cb52bf', 'request_id': 'req-81300d7b-d047-4ba4-8233-cd684255d82f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:30.350119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f161707b40e04d3089ca050e79cb52bf', 'request_id': 'req-81300d7b-d047-4ba4-8233-cd684255d82f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:30.357837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f161707b40e04d3089ca050e79cb52bf', 'request_id': 'req-81300d7b-d047-4ba4-8233-cd684255d82f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:30.400096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] The token cache is being invalidate because project f161707b40e04d3089ca050e79cb52bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:03:30.400396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:30.401126 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:30.405919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 672/1330] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:03:30 2022] DELETE /identity/v3/projects/f161707b40e04d3089ca050e79cb52bf => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:30.407262 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41309c0c-0d44-4683-90fa-5dc0e40d76c9 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:30.420964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41309c0c-0d44-4683-90fa-5dc0e40d76c9 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:30.619233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:30.620083 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:30.620465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:30.620839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] PATH_INFO: `/v3/users/be777ec5c77a4f3a8ddb4930f9bcdf35` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:30.624144 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be777ec5c77a4f3a8ddb4930f9bcdf35)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:30.627055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:30.642531 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:30.897220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be777ec5c77a4f3a8ddb4930f9bcdf35'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:30.898277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be777ec5c77a4f3a8ddb4930f9bcdf35'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:30.907235 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be777ec5c77a4f3a8ddb4930f9bcdf35'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:30.909767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be777ec5c77a4f3a8ddb4930f9bcdf35'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:30.995927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be777ec5c77a4f3a8ddb4930f9bcdf35'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:31.024871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 659/1331] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:03:30 2022] DELETE /identity/v3/users/be777ec5c77a4f3a8ddb4930f9bcdf35 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:31.452526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:31.454024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:31.478501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81300d7b-d047-4ba4-8233-cd684255d82f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:31.577412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:31.578376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:31.578867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:31.579363 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] PATH_INFO: `/v3/projects/71b9d960cf934d7483bb902bf3315922` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:31.580977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b9d960cf934d7483bb902bf3315922)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:31.583227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:31.604911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:31.610533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:31.640184 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854c6d5-cc16-426e-aa4d-5502f300bb49 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:31.673758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:31.677913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:31.677913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:31.677913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] PATH_INFO: `/v3/projects/8a33b99412544033a72f5aff6928917d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:31.687198 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a33b99412544033a72f5aff6928917d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:31.688517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:03:31.688517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Oct 15 13:03:31.688517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: Oct 15 13:03:31.690793 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:31.711647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:31.803299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b9d960cf934d7483bb902bf3315922', 'request_id': 'req-96d72cd1-b1ac-444c-a744-91f18e32b7a9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:31.903034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a33b99412544033a72f5aff6928917d', 'request_id': 'req-3400f8ab-da78-494a-804b-cd3c37d00a2c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:31.916487 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b9d960cf934d7483bb902bf3315922', 'request_id': 'req-96d72cd1-b1ac-444c-a744-91f18e32b7a9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:31.918811 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a33b99412544033a72f5aff6928917d', 'request_id': 'req-3400f8ab-da78-494a-804b-cd3c37d00a2c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:31.926425 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b9d960cf934d7483bb902bf3315922', 'request_id': 'req-96d72cd1-b1ac-444c-a744-91f18e32b7a9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:31.955377 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] The token cache is being invalidate because project 71b9d960cf934d7483bb902bf3315922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:03:31.955798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:31.956897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:31.963753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 673/1332] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:03:31 2022] DELETE /identity/v3/projects/71b9d960cf934d7483bb902bf3315922 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:31.964931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:31.966561 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:32.007564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a33b99412544033a72f5aff6928917d', 'request_id': 'req-3400f8ab-da78-494a-804b-cd3c37d00a2c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:32.025162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] The token cache is being invalidate because project 8a33b99412544033a72f5aff6928917d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:03:32.025698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:32.028618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3400f8ab-da78-494a-804b-cd3c37d00a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:32.030089 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 660/1333] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:03:31 2022] DELETE /identity/v3/projects/8a33b99412544033a72f5aff6928917d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:32.035487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078fca7-4668-4ced-99be-47f559470ee1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:32.036089 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078fca7-4668-4ced-99be-47f559470ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:32.036572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078fca7-4668-4ced-99be-47f559470ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:32.083072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d078fca7-4668-4ced-99be-47f559470ee1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:32.084819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d078fca7-4668-4ced-99be-47f559470ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:32.159267 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d72cd1-b1ac-444c-a744-91f18e32b7a9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:32.224814 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 661/1334] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:32.234806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87424d-3abd-49e6-ae31-feb818c94d05 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:32.235123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87424d-3abd-49e6-ae31-feb818c94d05 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:32.235418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87424d-3abd-49e6-ae31-feb818c94d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:32.259645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:32.261236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:32.261724 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:32.262328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] PATH_INFO: `/v3/users/19c19255331643d1ab51a6dd8aa009df` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:32.263892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c19255331643d1ab51a6dd8aa009df)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:32.264303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ed87424d-3abd-49e6-ae31-feb818c94d05 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:32.266319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ed87424d-3abd-49e6-ae31-feb818c94d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:32.268940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:32.280455 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 662/1335] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:03:32.287323 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87424d-3abd-49e6-ae31-feb818c94d05 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:32.328326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87424d-3abd-49e6-ae31-feb818c94d05 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:32.349221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e844-1bb3-4d10-abba-0ae8cb4ced70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:32.349221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e844-1bb3-4d10-abba-0ae8cb4ced70 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:32.349221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e844-1bb3-4d10-abba-0ae8cb4ced70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:32.349221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e844-1bb3-4d10-abba-0ae8cb4ced70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:32.349786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e844-1bb3-4d10-abba-0ae8cb4ced70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:32.350100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e844-1bb3-4d10-abba-0ae8cb4ced70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:32.351723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e844-1bb3-4d10-abba-0ae8cb4ced70 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:32.384942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 663/1336] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:32.390404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e844-1bb3-4d10-abba-0ae8cb4ced70 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:32.416827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e844-1bb3-4d10-abba-0ae8cb4ced70 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:32.439574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa603e-c4b2-4011-8f10-39a374670612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:32.441305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa603e-c4b2-4011-8f10-39a374670612 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:32.441764 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa603e-c4b2-4011-8f10-39a374670612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:32.442223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa603e-c4b2-4011-8f10-39a374670612 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:32.443488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa603e-c4b2-4011-8f10-39a374670612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:32.452399 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa603e-c4b2-4011-8f10-39a374670612 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:32.582847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:32.630011 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c19255331643d1ab51a6dd8aa009df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:32.636939 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 664/1337] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:32 2022] POST /identity/v3/projects => generated 381 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:32.643732 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa603e-c4b2-4011-8f10-39a374670612 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:32.653538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c19255331643d1ab51a6dd8aa009df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:32.655911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c19255331643d1ab51a6dd8aa009df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:32.660304 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c19255331643d1ab51a6dd8aa009df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:32.727261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c19255331643d1ab51a6dd8aa009df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:32.762730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 674/1338] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:03:31 2022] DELETE /identity/v3/users/19c19255331643d1ab51a6dd8aa009df => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:03:32.765735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:32.772963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:32.773770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa603e-c4b2-4011-8f10-39a374670612 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:32.878196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac62f7-1431-4171-ac65-52fed06755f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:32.881027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac62f7-1431-4171-ac65-52fed06755f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:32.882183 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac62f7-1431-4171-ac65-52fed06755f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:32.882569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac62f7-1431-4171-ac65-52fed06755f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:32.884125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac62f7-1431-4171-ac65-52fed06755f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:32.887649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac62f7-1431-4171-ac65-52fed06755f2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:32.918433 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2c397-fa0a-4c80-b834-17dde8ea55bd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:32.924979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:32.926266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:32.926813 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:32.927311 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] PATH_INFO: `/v3/projects/b16a2a5eb92c4940b20d21871d6abe4b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:32.928790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16a2a5eb92c4940b20d21871d6abe4b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:32.932128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:33.014097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 665/1339] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:32 2022] POST /identity/v3/users => generated 291 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:33.015345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac62f7-1431-4171-ac65-52fed06755f2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:33.019842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2fac62f7-1431-4171-ac65-52fed06755f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:33.055852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac62f7-1431-4171-ac65-52fed06755f2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:33.070458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16a2a5eb92c4940b20d21871d6abe4b', 'request_id': 'req-5adab047-0340-4c75-a023-bfdc7d718c25'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:33.134793 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:33.170852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16a2a5eb92c4940b20d21871d6abe4b', 'request_id': 'req-5adab047-0340-4c75-a023-bfdc7d718c25'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:33.184583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16a2a5eb92c4940b20d21871d6abe4b', 'request_id': 'req-5adab047-0340-4c75-a023-bfdc7d718c25'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:33.218962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f71e14-d1ec-4de5-903d-d66f76e069ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:33.219987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f71e14-d1ec-4de5-903d-d66f76e069ce admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.221796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f71e14-d1ec-4de5-903d-d66f76e069ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.222317 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f71e14-d1ec-4de5-903d-d66f76e069ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.223345 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f71e14-d1ec-4de5-903d-d66f76e069ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:33.226910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f71e14-d1ec-4de5-903d-d66f76e069ce admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:33.235466 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] The token cache is being invalidate because project b16a2a5eb92c4940b20d21871d6abe4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:03:33.238657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:33.242332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5adab047-0340-4c75-a023-bfdc7d718c25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:33.249399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 675/1340] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:03:32 2022] DELETE /identity/v3/projects/b16a2a5eb92c4940b20d21871d6abe4b => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:33.254075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 666/1341] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:03:33.255675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b0efd-1d6e-4d10-88c4-555f0c7d79d6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.255675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b0efd-1d6e-4d10-88c4-555f0c7d79d6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.255982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f71e14-d1ec-4de5-903d-d66f76e069ce admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:33.256089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b0efd-1d6e-4d10-88c4-555f0c7d79d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.261566 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b8f71e14-d1ec-4de5-903d-d66f76e069ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:33.290576 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f71e14-d1ec-4de5-903d-d66f76e069ce admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:33.301279 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7a6b0efd-1d6e-4d10-88c4-555f0c7d79d6 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:33.302677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7a6b0efd-1d6e-4d10-88c4-555f0c7d79d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:33.302972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7404d-d120-4fae-8712-ceb7a8c9b7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:33.304792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7404d-d120-4fae-8712-ceb7a8c9b7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.305080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7404d-d120-4fae-8712-ceb7a8c9b7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.307560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7404d-d120-4fae-8712-ceb7a8c9b7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.308490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7404d-d120-4fae-8712-ceb7a8c9b7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:33.312106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7404d-d120-4fae-8712-ceb7a8c9b7ca admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:33.333082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 676/1342] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:33.341814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0b986-1219-48c5-989b-23a8fc224e77 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.346479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 667/1343] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:33.346479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7404d-d120-4fae-8712-ceb7a8c9b7ca admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:33.348624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0b986-1219-48c5-989b-23a8fc224e77 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.349114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0b986-1219-48c5-989b-23a8fc224e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.371038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7404d-d120-4fae-8712-ceb7a8c9b7ca admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:33.376866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d09d84-43d2-4191-9b7f-1b58c0eb5ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:33.379028 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d09d84-43d2-4191-9b7f-1b58c0eb5ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.380720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d09d84-43d2-4191-9b7f-1b58c0eb5ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.380720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d09d84-43d2-4191-9b7f-1b58c0eb5ddd admin admin] PATH_INFO: `/v3/projects/631e6b22fa1b4166a4371776687fffe6/users/c4ab77eb28d746ceb2824340614d2167/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.400872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0da0b986-1219-48c5-989b-23a8fc224e77 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:33.402520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0da0b986-1219-48c5-989b-23a8fc224e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:33.413070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d09d84-43d2-4191-9b7f-1b58c0eb5ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=631e6b22fa1b4166a4371776687fffe6, user_id=c4ab77eb28d746ceb2824340614d2167, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:33.417422 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 677/1344] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:33.417555 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d09d84-43d2-4191-9b7f-1b58c0eb5ddd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:33.418730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0b986-1219-48c5-989b-23a8fc224e77 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:33.435320 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0b986-1219-48c5-989b-23a8fc224e77 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:33.457268 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9839137-13d8-4f36-98e4-11502a1a6f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:33.458248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839137-13d8-4f36-98e4-11502a1a6f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.459008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839137-13d8-4f36-98e4-11502a1a6f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.459534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839137-13d8-4f36-98e4-11502a1a6f42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.460415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9839137-13d8-4f36-98e4-11502a1a6f42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:33.460900 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9839137-13d8-4f36-98e4-11502a1a6f42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:33.463466 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9839137-13d8-4f36-98e4-11502a1a6f42 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:33.487298 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 678/1345] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:33.491455 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9839137-13d8-4f36-98e4-11502a1a6f42 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:33.503921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9839137-13d8-4f36-98e4-11502a1a6f42 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:33.513414 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 668/1346] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:33 2022] PUT /identity/v3/projects/631e6b22fa1b4166a4371776687fffe6/users/c4ab77eb28d746ceb2824340614d2167/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:33.515741 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646225a6-8973-42a0-b957-d09ed08cd7ac None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.516044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646225a6-8973-42a0-b957-d09ed08cd7ac None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.516923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646225a6-8973-42a0-b957-d09ed08cd7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.598264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:33.599782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-646225a6-8973-42a0-b957-d09ed08cd7ac None None] MFA Rules not processed for user `c4ab77eb28d746ceb2824340614d2167`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:33.602205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-646225a6-8973-42a0-b957-d09ed08cd7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:33.636877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c577aa-425f-4ea4-b69a-5bad6cf31b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:33.637623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c577aa-425f-4ea4-b69a-5bad6cf31b66 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.637910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c577aa-425f-4ea4-b69a-5bad6cf31b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.638188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c577aa-425f-4ea4-b69a-5bad6cf31b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.638841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c577aa-425f-4ea4-b69a-5bad6cf31b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:33.641069 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c577aa-425f-4ea4-b69a-5bad6cf31b66 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:33.757875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 679/1347] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:33 2022] POST /identity/v3/projects => generated 391 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:33.759485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 669/1348] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:33.763629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c577aa-425f-4ea4-b69a-5bad6cf31b66 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:33.764390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646225a6-8973-42a0-b957-d09ed08cd7ac None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:33.785405 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646225a6-8973-42a0-b957-d09ed08cd7ac None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:33.903153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c577aa-425f-4ea4-b69a-5bad6cf31b66 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:33.908972 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918e045-12ec-4fc5-9e59-ba42d02db5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:33.911762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918e045-12ec-4fc5-9e59-ba42d02db5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.911762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918e045-12ec-4fc5-9e59-ba42d02db5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.911762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918e045-12ec-4fc5-9e59-ba42d02db5c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.911762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918e045-12ec-4fc5-9e59-ba42d02db5c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:33.912725 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918e045-12ec-4fc5-9e59-ba42d02db5c3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:33.925838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709f5df-4d08-4260-a2d8-919729783b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:33.927578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709f5df-4d08-4260-a2d8-919729783b55 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:33.929497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709f5df-4d08-4260-a2d8-919729783b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:33.929956 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709f5df-4d08-4260-a2d8-919729783b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:33.931564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709f5df-4d08-4260-a2d8-919729783b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:33.934973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709f5df-4d08-4260-a2d8-919729783b55 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:34.029656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 680/1349] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:33 2022] POST /identity/v3/users => generated 296 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:34.036233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918e045-12ec-4fc5-9e59-ba42d02db5c3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:34.074504 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918e045-12ec-4fc5-9e59-ba42d02db5c3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:34.084743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bb85e-680c-41d1-a9e4-d8f2743b8a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:34.086045 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bb85e-680c-41d1-a9e4-d8f2743b8a93 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:34.086299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bb85e-680c-41d1-a9e4-d8f2743b8a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:34.086629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bb85e-680c-41d1-a9e4-d8f2743b8a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:34.087866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bb85e-680c-41d1-a9e4-d8f2743b8a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:34.089381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 670/1350] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:33 2022] POST /identity/v3/projects => generated 379 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:34.093996 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709f5df-4d08-4260-a2d8-919729783b55 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:34.094629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bb85e-680c-41d1-a9e4-d8f2743b8a93 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:34.111171 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 681/1351] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:34.115682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bb85e-680c-41d1-a9e4-d8f2743b8a93 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:34.131546 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bb85e-680c-41d1-a9e4-d8f2743b8a93 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:34.146447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:34.228305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:34.256433 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd04d97-1808-4109-a792-d02927852bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:34.257079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd04d97-1808-4109-a792-d02927852bea admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:34.257390 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd04d97-1808-4109-a792-d02927852bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:34.257670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd04d97-1808-4109-a792-d02927852bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:34.258338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd04d97-1808-4109-a792-d02927852bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:34.259989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd04d97-1808-4109-a792-d02927852bea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:34.268240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709f5df-4d08-4260-a2d8-919729783b55 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:34.286196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c390392-e298-4243-8fdd-cf78dc5196cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:34.286196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c390392-e298-4243-8fdd-cf78dc5196cf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:34.286196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c390392-e298-4243-8fdd-cf78dc5196cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:34.286196 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c390392-e298-4243-8fdd-cf78dc5196cf admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:34.286622 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c390392-e298-4243-8fdd-cf78dc5196cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:34.287509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 682/1352] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:34.288421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd04d97-1808-4109-a792-d02927852bea admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:34.290149 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c390392-e298-4243-8fdd-cf78dc5196cf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:34.312186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd04d97-1808-4109-a792-d02927852bea admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:34.320256 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bc350-57b3-47e8-8630-2b434becce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:34.320947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bc350-57b3-47e8-8630-2b434becce5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:34.321248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bc350-57b3-47e8-8630-2b434becce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:34.321527 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bc350-57b3-47e8-8630-2b434becce5d admin admin] PATH_INFO: `/v3/projects/7f5821d5735840b499a3e954a2ad1319/users/edc1b2c441034889a7625bd5f3f4c042/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:34.341277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bc350-57b3-47e8-8630-2b434becce5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5821d5735840b499a3e954a2ad1319, user_id=edc1b2c441034889a7625bd5f3f4c042, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:34.344952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bc350-57b3-47e8-8630-2b434becce5d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:34.436533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 671/1353] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:34 2022] POST /identity/v3/users => generated 296 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:34.440716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c390392-e298-4243-8fdd-cf78dc5196cf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:34.449399 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0c390392-e298-4243-8fdd-cf78dc5196cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:34.459098 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 683/1354] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:34 2022] PUT /identity/v3/projects/7f5821d5735840b499a3e954a2ad1319/users/edc1b2c441034889a7625bd5f3f4c042/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:34.459098 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bc350-57b3-47e8-8630-2b434becce5d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:34.472497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c390392-e298-4243-8fdd-cf78dc5196cf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:34.534996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bc350-57b3-47e8-8630-2b434becce5d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:34.545995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a3980-8489-4c35-a295-bd3eb70c303f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:34.549804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a3980-8489-4c35-a295-bd3eb70c303f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:34.549804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a3980-8489-4c35-a295-bd3eb70c303f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:34.549804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a3980-8489-4c35-a295-bd3eb70c303f admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:34.549804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a3980-8489-4c35-a295-bd3eb70c303f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:34.552529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a3980-8489-4c35-a295-bd3eb70c303f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:34.595170 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 684/1355] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:34.599334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db98455-a1dd-4af4-af51-fea09eeeee02 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:34.600814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db98455-a1dd-4af4-af51-fea09eeeee02 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:34.602495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db98455-a1dd-4af4-af51-fea09eeeee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:34.637137 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:34.641022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:34.641022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:34.641022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] PATH_INFO: `/v3/projects/f3c7052b73664cdf93b0bd738ce2b5f7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:34.642118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3c7052b73664cdf93b0bd738ce2b5f7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:34.646287 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:34.682937 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5db98455-a1dd-4af4-af51-fea09eeeee02 None None] MFA Rules not processed for user `edc1b2c441034889a7625bd5f3f4c042`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:34.684333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5db98455-a1dd-4af4-af51-fea09eeeee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:34.805502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3c7052b73664cdf93b0bd738ce2b5f7', 'request_id': 'req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:34.811012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 685/1356] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:34.812179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db98455-a1dd-4af4-af51-fea09eeeee02 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:34.838152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3c7052b73664cdf93b0bd738ce2b5f7', 'request_id': 'req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:34.844028 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db98455-a1dd-4af4-af51-fea09eeeee02 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:34.928112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3c7052b73664cdf93b0bd738ce2b5f7', 'request_id': 'req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:34.948557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] The token cache is being invalidate because project f3c7052b73664cdf93b0bd738ce2b5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:03:34.949708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:34.953569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:34.956523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 672/1357] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:03:34 2022] DELETE /identity/v3/projects/f3c7052b73664cdf93b0bd738ce2b5f7 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:34.958319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:34.963606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:35.017532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d17ea3-c5e3-489b-b6c0-b0a0834ce590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:35.017532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d17ea3-c5e3-489b-b6c0-b0a0834ce590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.017532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d17ea3-c5e3-489b-b6c0-b0a0834ce590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.021224 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d17ea3-c5e3-489b-b6c0-b0a0834ce590 admin admin] PATH_INFO: `/v3/projects/02200fe4fa3944ac8f06679c7dd4b190/users/9fcc6fabb2a04de7bb3b93cfc23013e7/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.061652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d17ea3-c5e3-489b-b6c0-b0a0834ce590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02200fe4fa3944ac8f06679c7dd4b190, user_id=9fcc6fabb2a04de7bb3b93cfc23013e7, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:35.069650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d17ea3-c5e3-489b-b6c0-b0a0834ce590 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:35.152632 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 686/1358] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:34 2022] PUT /identity/v3/projects/02200fe4fa3944ac8f06679c7dd4b190/users/9fcc6fabb2a04de7bb3b93cfc23013e7/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:35.155036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a32a0c-87c1-49f0-8738-cdf6f8ba1220 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.155540 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a32a0c-87c1-49f0-8738-cdf6f8ba1220 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.158255 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a32a0c-87c1-49f0-8738-cdf6f8ba1220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.167791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910cb5-5684-4e65-a0f3-2f338e1a1ec9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:35.189012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c1a32a0c-87c1-49f0-8738-cdf6f8ba1220 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:35.191007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c1a32a0c-87c1-49f0-8738-cdf6f8ba1220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:35.239526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:35.287960 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 687/1359] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:35.294560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a32a0c-87c1-49f0-8738-cdf6f8ba1220 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:35.297585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c1a32a0c-87c1-49f0-8738-cdf6f8ba1220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:35.327234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bbfe4-426a-4ee6-95b5-641a73f1b779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:35.328083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a32a0c-87c1-49f0-8738-cdf6f8ba1220 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:35.328433 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bbfe4-426a-4ee6-95b5-641a73f1b779 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.329055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bbfe4-426a-4ee6-95b5-641a73f1b779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.330454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bbfe4-426a-4ee6-95b5-641a73f1b779 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.331467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bbfe4-426a-4ee6-95b5-641a73f1b779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:35.333912 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bbfe4-426a-4ee6-95b5-641a73f1b779 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:35.334651 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006cb4f-d43b-48a5-84a3-67b68b4fa8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:35.335430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006cb4f-d43b-48a5-84a3-67b68b4fa8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.335828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006cb4f-d43b-48a5-84a3-67b68b4fa8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.336210 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006cb4f-d43b-48a5-84a3-67b68b4fa8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.337315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006cb4f-d43b-48a5-84a3-67b68b4fa8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:35.339872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006cb4f-d43b-48a5-84a3-67b68b4fa8e7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:35.376954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 688/1360] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:35.380261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4be4c-6117-4946-ab13-7304fc80012f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.380261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4be4c-6117-4946-ab13-7304fc80012f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.380261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4be4c-6117-4946-ab13-7304fc80012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.411846 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2fa4be4c-6117-4946-ab13-7304fc80012f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:35.414951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2fa4be4c-6117-4946-ab13-7304fc80012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:35.426300 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:35.431328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 689/1361] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:35.432416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4be4c-6117-4946-ab13-7304fc80012f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:35.446580 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4be4c-6117-4946-ab13-7304fc80012f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:35.454053 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a9ee1-9244-4bf7-9f63-64a10e292ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:35.455198 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a9ee1-9244-4bf7-9f63-64a10e292ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.456331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a9ee1-9244-4bf7-9f63-64a10e292ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.456805 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a9ee1-9244-4bf7-9f63-64a10e292ee3 admin admin] PATH_INFO: `/v3/projects/02200fe4fa3944ac8f06679c7dd4b190/users/9fcc6fabb2a04de7bb3b93cfc23013e7/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.461285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a9ee1-9244-4bf7-9f63-64a10e292ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02200fe4fa3944ac8f06679c7dd4b190, user_id=9fcc6fabb2a04de7bb3b93cfc23013e7, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:35.474619 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a9ee1-9244-4bf7-9f63-64a10e292ee3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:35.497876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 673/1362] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:34 2022] POST /identity/v3/projects => generated 391 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:35.498935 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bbfe4-426a-4ee6-95b5-641a73f1b779 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:35.516984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 690/1363] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:35 2022] PUT /identity/v3/projects/02200fe4fa3944ac8f06679c7dd4b190/users/9fcc6fabb2a04de7bb3b93cfc23013e7/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:35.518610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a9ee1-9244-4bf7-9f63-64a10e292ee3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:35.663562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bbfe4-426a-4ee6-95b5-641a73f1b779 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:35.681758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb265a-2562-4cd5-8dc9-ac8e0c4988fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:35.683114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb265a-2562-4cd5-8dc9-ac8e0c4988fc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.684152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb265a-2562-4cd5-8dc9-ac8e0c4988fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.685441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb265a-2562-4cd5-8dc9-ac8e0c4988fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.689384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb265a-2562-4cd5-8dc9-ac8e0c4988fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:03:35.690766 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb265a-2562-4cd5-8dc9-ac8e0c4988fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:35.692378 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb265a-2562-4cd5-8dc9-ac8e0c4988fc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:35.737607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a9ee1-9244-4bf7-9f63-64a10e292ee3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:35.743082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5a86b-374e-4ef8-a46e-952cde9df859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:35.743714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5a86b-374e-4ef8-a46e-952cde9df859 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.743993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5a86b-374e-4ef8-a46e-952cde9df859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.750332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5a86b-374e-4ef8-a46e-952cde9df859 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.750332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5a86b-374e-4ef8-a46e-952cde9df859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:35.750332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5a86b-374e-4ef8-a46e-952cde9df859 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:35.750638 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 674/1364] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:03:35.751513 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423c3d4-1992-4aa8-8830-b287377f5be6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.754151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423c3d4-1992-4aa8-8830-b287377f5be6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.756687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423c3d4-1992-4aa8-8830-b287377f5be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.836332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 691/1365] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:35 2022] POST /identity/v3/users => generated 302 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:35.836332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5a86b-374e-4ef8-a46e-952cde9df859 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:35.840484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2423c3d4-1992-4aa8-8830-b287377f5be6 None None] MFA Rules not processed for user `9fcc6fabb2a04de7bb3b93cfc23013e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:35.843158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2423c3d4-1992-4aa8-8830-b287377f5be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:35.868335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5a86b-374e-4ef8-a46e-952cde9df859 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:35.889498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ad112-438e-4f9f-a8cd-f30c5a177cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:35.890497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ad112-438e-4f9f-a8cd-f30c5a177cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:35.890786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ad112-438e-4f9f-a8cd-f30c5a177cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:35.891062 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ad112-438e-4f9f-a8cd-f30c5a177cf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:35.896116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ad112-438e-4f9f-a8cd-f30c5a177cf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:35.902671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ad112-438e-4f9f-a8cd-f30c5a177cf2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:36.045675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 692/1366] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:35 2022] POST /identity/v3/projects => generated 357 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:36.046602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ad112-438e-4f9f-a8cd-f30c5a177cf2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:36.101455 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 675/1367] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:36.102360 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423c3d4-1992-4aa8-8830-b287377f5be6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:36.140860 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423c3d4-1992-4aa8-8830-b287377f5be6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:36.279573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771fd2b-431f-43cf-a43e-c6e2274f9f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:36.280355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771fd2b-431f-43cf-a43e-c6e2274f9f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:36.280646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771fd2b-431f-43cf-a43e-c6e2274f9f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:36.280924 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771fd2b-431f-43cf-a43e-c6e2274f9f30 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:36.281673 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771fd2b-431f-43cf-a43e-c6e2274f9f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:36.284505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ad112-438e-4f9f-a8cd-f30c5a177cf2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:36.287584 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771fd2b-431f-43cf-a43e-c6e2274f9f30 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:36.304569 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43f9e5-dcf8-4e24-aa0f-01da3c07ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:36.307427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43f9e5-dcf8-4e24-aa0f-01da3c07ffc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:36.307717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43f9e5-dcf8-4e24-aa0f-01da3c07ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:36.307999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43f9e5-dcf8-4e24-aa0f-01da3c07ffc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:36.308719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43f9e5-dcf8-4e24-aa0f-01da3c07ffc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:36.312610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43f9e5-dcf8-4e24-aa0f-01da3c07ffc8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:36.330486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 693/1368] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:36.334772 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43f9e5-dcf8-4e24-aa0f-01da3c07ffc8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:36.340421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2b43f9e5-dcf8-4e24-aa0f-01da3c07ffc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:36.417888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 676/1369] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:36 2022] POST /identity/v3/users => generated 279 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:03:36.417888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771fd2b-431f-43cf-a43e-c6e2274f9f30 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:36.450755 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771fd2b-431f-43cf-a43e-c6e2274f9f30 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:36.491429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36df8-1d9a-411b-af08-9ea9f05f92b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:36.492147 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36df8-1d9a-411b-af08-9ea9f05f92b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:36.492437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36df8-1d9a-411b-af08-9ea9f05f92b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:36.492719 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36df8-1d9a-411b-af08-9ea9f05f92b1 admin admin] PATH_INFO: `/v3/projects/7cb39bc6e8a74d90a7794a695af33cfd/users/0be1a13741fe42a2ab8845d0b452fac8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:36.525494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea36df8-1d9a-411b-af08-9ea9f05f92b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb39bc6e8a74d90a7794a695af33cfd, user_id=0be1a13741fe42a2ab8845d0b452fac8, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:36.538701 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:36.538843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea36df8-1d9a-411b-af08-9ea9f05f92b1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:36.560453 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43f9e5-dcf8-4e24-aa0f-01da3c07ffc8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:36.584988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ad286-fa0c-4a70-949b-6f26a05ac059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:36.584988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ad286-fa0c-4a70-949b-6f26a05ac059 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:36.584988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ad286-fa0c-4a70-949b-6f26a05ac059 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:36.584988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ad286-fa0c-4a70-949b-6f26a05ac059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:36.669692 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 677/1370] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:36 2022] PUT /identity/v3/projects/7cb39bc6e8a74d90a7794a695af33cfd/users/0be1a13741fe42a2ab8845d0b452fac8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:36.670757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36df8-1d9a-411b-af08-9ea9f05f92b1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:36.742157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:36.751628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ad286-fa0c-4a70-949b-6f26a05ac059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:36.753093 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ad286-fa0c-4a70-949b-6f26a05ac059 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:36.791336 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 694/1371] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:36.794654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ad286-fa0c-4a70-949b-6f26a05ac059 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:36.824480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ad286-fa0c-4a70-949b-6f26a05ac059 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:36.934910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36df8-1d9a-411b-af08-9ea9f05f92b1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:36.968648 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d1a9d-5491-49e1-9a32-ed1631b15ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:36.969369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1a9d-5491-49e1-9a32-ed1631b15ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:36.969651 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1a9d-5491-49e1-9a32-ed1631b15ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:36.969925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1a9d-5491-49e1-9a32-ed1631b15ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:36.973695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d1a9d-5491-49e1-9a32-ed1631b15ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:36.993118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d1a9d-5491-49e1-9a32-ed1631b15ba6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:37.066937 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 678/1372] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:37.072835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d1a9d-5491-49e1-9a32-ed1631b15ba6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:37.095152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d1a9d-5491-49e1-9a32-ed1631b15ba6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:37.100901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3792acb-ea77-44a5-8785-43d861ea1e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:37.103826 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3792acb-ea77-44a5-8785-43d861ea1e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:37.104134 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3792acb-ea77-44a5-8785-43d861ea1e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:37.104413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3792acb-ea77-44a5-8785-43d861ea1e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:37.109115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3792acb-ea77-44a5-8785-43d861ea1e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:37.124038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d85a41-9f09-4890-8ca6-65e424232d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:37.125242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d85a41-9f09-4890-8ca6-65e424232d0a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:37.125721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d85a41-9f09-4890-8ca6-65e424232d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:37.126147 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d85a41-9f09-4890-8ca6-65e424232d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:37.126968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d85a41-9f09-4890-8ca6-65e424232d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:37.130453 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d85a41-9f09-4890-8ca6-65e424232d0a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:37.130610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3792acb-ea77-44a5-8785-43d861ea1e88 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:37.145100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 695/1373] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:37.145100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3792acb-ea77-44a5-8785-43d861ea1e88 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:37.177841 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 679/1374] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:37.182822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d85a41-9f09-4890-8ca6-65e424232d0a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:37.210465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d85a41-9f09-4890-8ca6-65e424232d0a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:37.217710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607169-c4a9-4f16-80ed-28c433d52ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:37.219046 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607169-c4a9-4f16-80ed-28c433d52ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:37.219443 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607169-c4a9-4f16-80ed-28c433d52ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:37.219777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607169-c4a9-4f16-80ed-28c433d52ad9 admin admin] PATH_INFO: `/v3/projects/7cb39bc6e8a74d90a7794a695af33cfd/users/0be1a13741fe42a2ab8845d0b452fac8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:37.223959 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea607169-c4a9-4f16-80ed-28c433d52ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb39bc6e8a74d90a7794a695af33cfd, user_id=0be1a13741fe42a2ab8845d0b452fac8, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:37.235918 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea607169-c4a9-4f16-80ed-28c433d52ad9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:37.269327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 680/1375] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:37 2022] PUT /identity/v3/projects/7cb39bc6e8a74d90a7794a695af33cfd/users/0be1a13741fe42a2ab8845d0b452fac8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:37.271029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607169-c4a9-4f16-80ed-28c433d52ad9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:37.290399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3792acb-ea77-44a5-8785-43d861ea1e88 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:37.421691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607169-c4a9-4f16-80ed-28c433d52ad9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:37.425111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8648f6-e845-423e-815a-dd12d427232c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:37.426139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8648f6-e845-423e-815a-dd12d427232c service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:37.434941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8648f6-e845-423e-815a-dd12d427232c service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:37.434941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8648f6-e845-423e-815a-dd12d427232c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:37.434941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ad8648f6-e845-423e-815a-dd12d427232c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:37.435266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece9e73-a18d-448e-b780-8b92c1acb8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:37.435266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9e73-a18d-448e-b780-8b92c1acb8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:37.435266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9e73-a18d-448e-b780-8b92c1acb8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:37.435266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9e73-a18d-448e-b780-8b92c1acb8f7 admin admin] PATH_INFO: `/v3/projects/c866b8e05ef84edbae2b1a0baf04f3a3/users/5196272eaf0846a186088131daa36cc9/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:37.461799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece9e73-a18d-448e-b780-8b92c1acb8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c866b8e05ef84edbae2b1a0baf04f3a3, user_id=5196272eaf0846a186088131daa36cc9, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:37.466047 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece9e73-a18d-448e-b780-8b92c1acb8f7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:37.529341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 681/1376] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:37 2022] PUT /identity/v3/projects/c866b8e05ef84edbae2b1a0baf04f3a3/users/5196272eaf0846a186088131daa36cc9/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:37.531425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4357bf4-7d52-4601-a6cc-57a096c455b2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:37.531808 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4357bf4-7d52-4601-a6cc-57a096c455b2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:37.532350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4357bf4-7d52-4601-a6cc-57a096c455b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:37.545170 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:37.581608 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8648f6-e845-423e-815a-dd12d427232c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:37.583098 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8648f6-e845-423e-815a-dd12d427232c service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:37.607854 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 696/1377] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:37.609355 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75988647-423f-4046-b50d-49c2594c42f6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:37.609786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75988647-423f-4046-b50d-49c2594c42f6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:37.610206 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75988647-423f-4046-b50d-49c2594c42f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:37.613177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a4357bf4-7d52-4601-a6cc-57a096c455b2 None None] MFA Rules not processed for user `0be1a13741fe42a2ab8845d0b452fac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:37.615501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a4357bf4-7d52-4601-a6cc-57a096c455b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:37.642903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:37.678396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-75988647-423f-4046-b50d-49c2594c42f6 None None] MFA Rules not processed for user `5196272eaf0846a186088131daa36cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:37.680033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-75988647-423f-4046-b50d-49c2594c42f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:37.820152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 682/1378] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:37 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:37.828388 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4357bf4-7d52-4601-a6cc-57a096c455b2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:37.830812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a4357bf4-7d52-4601-a6cc-57a096c455b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:37.887375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 697/1379] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:37.892416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75988647-423f-4046-b50d-49c2594c42f6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:37.895695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:37.961243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4357bf4-7d52-4601-a6cc-57a096c455b2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:37.966641 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c4e4-34d6-4feb-9dbd-0a4588cfd85b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:37.967289 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c4e4-34d6-4feb-9dbd-0a4588cfd85b service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:37.967569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c4e4-34d6-4feb-9dbd-0a4588cfd85b service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:37.967844 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c4e4-34d6-4feb-9dbd-0a4588cfd85b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:37.995243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206c4e4-34d6-4feb-9dbd-0a4588cfd85b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:37.998915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206c4e4-34d6-4feb-9dbd-0a4588cfd85b service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:38.018386 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 683/1380] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:37 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:03:38.102195 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75988647-423f-4046-b50d-49c2594c42f6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:38.116925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7698b1-9f4b-438e-8014-0da21069e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:38.120844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7698b1-9f4b-438e-8014-0da21069e988 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:38.123199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7698b1-9f4b-438e-8014-0da21069e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:38.124689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7698b1-9f4b-438e-8014-0da21069e988 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:38.125590 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7698b1-9f4b-438e-8014-0da21069e988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:38.133537 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7698b1-9f4b-438e-8014-0da21069e988 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:38.190399 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c4e4-34d6-4feb-9dbd-0a4588cfd85b service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:38.213838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c4e4-34d6-4feb-9dbd-0a4588cfd85b service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:38.229678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668334f-c81b-4490-857e-1bf3854a54b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:38.230315 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668334f-c81b-4490-857e-1bf3854a54b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:38.230458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:38.230596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668334f-c81b-4490-857e-1bf3854a54b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:38.230874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668334f-c81b-4490-857e-1bf3854a54b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:38.238714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3668334f-c81b-4490-857e-1bf3854a54b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:38.307801 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 698/1381] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:03:37 2022] POST /identity/v3/projects => generated 355 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:03:38.312039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7698b1-9f4b-438e-8014-0da21069e988 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:38.452263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668334f-c81b-4490-857e-1bf3854a54b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:38.453857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668334f-c81b-4490-857e-1bf3854a54b4 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:38.461010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:38.469098 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 684/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:03:38.507680 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7698b1-9f4b-438e-8014-0da21069e988 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:38.513108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71611a49-c7c5-4dfc-9f57-a8d722ff31de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:38.514231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71611a49-c7c5-4dfc-9f57-a8d722ff31de admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:38.514454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71611a49-c7c5-4dfc-9f57-a8d722ff31de admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:38.514730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71611a49-c7c5-4dfc-9f57-a8d722ff31de admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:38.515376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71611a49-c7c5-4dfc-9f57-a8d722ff31de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:38.517378 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71611a49-c7c5-4dfc-9f57-a8d722ff31de admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:38.623275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 699/1383] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:03:38 2022] POST /identity/v3/users => generated 284 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:03:38.627790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668334f-c81b-4490-857e-1bf3854a54b4 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:38.651517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668334f-c81b-4490-857e-1bf3854a54b4 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:38.675373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837efa87-b1a2-4f15-8c4f-387e41520c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:38.681672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837efa87-b1a2-4f15-8c4f-387e41520c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:38.681962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837efa87-b1a2-4f15-8c4f-387e41520c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:38.682242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837efa87-b1a2-4f15-8c4f-387e41520c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:38.682929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837efa87-b1a2-4f15-8c4f-387e41520c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:38.684560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837efa87-b1a2-4f15-8c4f-387e41520c4a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:38.709754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 685/1384] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:38.713717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71611a49-c7c5-4dfc-9f57-a8d722ff31de admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:38.728039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71611a49-c7c5-4dfc-9f57-a8d722ff31de admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:38.751020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49041-1b74-4928-b366-5d33b5a41c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:38.751020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49041-1b74-4928-b366-5d33b5a41c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:38.751020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49041-1b74-4928-b366-5d33b5a41c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:38.751020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49041-1b74-4928-b366-5d33b5a41c89 admin admin] PATH_INFO: `/v3/projects/44183268008e4dd39615e350aa86c56d/users/54448e6c6de44e87b0994365067664ab/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:38.782981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a49041-1b74-4928-b366-5d33b5a41c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44183268008e4dd39615e350aa86c56d, user_id=54448e6c6de44e87b0994365067664ab, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:38.789740 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a49041-1b74-4928-b366-5d33b5a41c89 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:38.895717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 700/1385] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:38 2022] PUT /identity/v3/projects/44183268008e4dd39615e350aa86c56d/users/54448e6c6de44e87b0994365067664ab/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:03:38.899943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837efa87-b1a2-4f15-8c4f-387e41520c4a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:38.914290 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837efa87-b1a2-4f15-8c4f-387e41520c4a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:39.054753 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f7d43-6312-49e0-8078-763346283f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:39.055650 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7d43-6312-49e0-8078-763346283f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:39.060416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7d43-6312-49e0-8078-763346283f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:39.060947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7d43-6312-49e0-8078-763346283f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:39.064709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f7d43-6312-49e0-8078-763346283f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:39.067012 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f7d43-6312-49e0-8078-763346283f79 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:39.086750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 686/1386] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:03:39.087289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49041-1b74-4928-b366-5d33b5a41c89 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:39.106013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49041-1b74-4928-b366-5d33b5a41c89 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:39.118274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be3f48-b47a-4c3e-a801-b7978da8877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:39.119141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be3f48-b47a-4c3e-a801-b7978da8877c admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:39.119496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be3f48-b47a-4c3e-a801-b7978da8877c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:39.133438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be3f48-b47a-4c3e-a801-b7978da8877c admin admin] PATH_INFO: `/v3/projects/44183268008e4dd39615e350aa86c56d/users/54448e6c6de44e87b0994365067664ab/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:39.143156 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be3f48-b47a-4c3e-a801-b7978da8877c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44183268008e4dd39615e350aa86c56d, user_id=54448e6c6de44e87b0994365067664ab, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:39.151327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be3f48-b47a-4c3e-a801-b7978da8877c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:39.199824 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 701/1387] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:03:39 2022] PUT /identity/v3/projects/44183268008e4dd39615e350aa86c56d/users/54448e6c6de44e87b0994365067664ab/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:39.214519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d344cd-6104-483c-87a1-ce9234e04510 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:39.214883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d344cd-6104-483c-87a1-ce9234e04510 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:39.215178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d344cd-6104-483c-87a1-ce9234e04510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:39.306956 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f0d344cd-6104-483c-87a1-ce9234e04510 None None] MFA Rules not processed for user `54448e6c6de44e87b0994365067664ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:39.308262 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f0d344cd-6104-483c-87a1-ce9234e04510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:39.417542 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4741863-b589-4b10-a6d1-cbe713c77241 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:39.417661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4741863-b589-4b10-a6d1-cbe713c77241 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:39.417965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4741863-b589-4b10-a6d1-cbe713c77241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:39.562624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 687/1388] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:39.571258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e4741863-b589-4b10-a6d1-cbe713c77241 None None] MFA Rules not processed for user `83bf124cd05147718d1d7a062fed7c64`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:39.573764 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e4741863-b589-4b10-a6d1-cbe713c77241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:39.624492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d344cd-6104-483c-87a1-ce9234e04510 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:39.663635 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:39.755060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d344cd-6104-483c-87a1-ce9234e04510 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:39.759791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c402f14-9c00-4798-b243-aedf29639e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:39.760423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c402f14-9c00-4798-b243-aedf29639e14 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:39.760708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c402f14-9c00-4798-b243-aedf29639e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:39.760986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c402f14-9c00-4798-b243-aedf29639e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:39.784352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c402f14-9c00-4798-b243-aedf29639e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:39.790457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c402f14-9c00-4798-b243-aedf29639e14 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:39.792210 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 702/1389] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:03:39.801883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe86986-7729-4459-8e07-b1f9da8eea6e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:39.802174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe86986-7729-4459-8e07-b1f9da8eea6e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:39.802461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe86986-7729-4459-8e07-b1f9da8eea6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:39.829041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 688/1390] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:39 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:39.853490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7fe86986-7729-4459-8e07-b1f9da8eea6e None None] MFA Rules not processed for user `83bf124cd05147718d1d7a062fed7c64`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:39.855009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7fe86986-7729-4459-8e07-b1f9da8eea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:39.886691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 703/1391] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:39.895455 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c402f14-9c00-4798-b243-aedf29639e14 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:39.896928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2c402f14-9c00-4798-b243-aedf29639e14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:39.940269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:40.151954 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c402f14-9c00-4798-b243-aedf29639e14 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:40.174875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11453cc-9f46-4855-80e1-11c73aba8139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:40.175563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11453cc-9f46-4855-80e1-11c73aba8139 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:40.176114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11453cc-9f46-4855-80e1-11c73aba8139 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:40.176392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11453cc-9f46-4855-80e1-11c73aba8139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:40.213232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11453cc-9f46-4855-80e1-11c73aba8139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:40.214930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11453cc-9f46-4855-80e1-11c73aba8139 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:40.243437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 689/1392] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:40.481869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe86986-7729-4459-8e07-b1f9da8eea6e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:40.501387 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe86986-7729-4459-8e07-b1f9da8eea6e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:40.517384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c1be-e87a-4458-baf3-ab403a4daa2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:40.518747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c1be-e87a-4458-baf3-ab403a4daa2a service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:40.519032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c1be-e87a-4458-baf3-ab403a4daa2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:40.519403 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c1be-e87a-4458-baf3-ab403a4daa2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:40.742159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5c1be-e87a-4458-baf3-ab403a4daa2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:40.749265 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5c1be-e87a-4458-baf3-ab403a4daa2a service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:40.778452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 704/1393] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:40 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:41.065219 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad90f9f-5592-4708-b767-e3bf92475673 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:41.065735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad90f9f-5592-4708-b767-e3bf92475673 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:41.066000 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad90f9f-5592-4708-b767-e3bf92475673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:41.166615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cad90f9f-5592-4708-b767-e3bf92475673 None None] MFA Rules not processed for user `cfbb59c3a18a4bbaa752cbee63ae128d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:41.170341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cad90f9f-5592-4708-b767-e3bf92475673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:41.369738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 690/1394] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:41.390466 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c1be-e87a-4458-baf3-ab403a4daa2a service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:41.391880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1ff5c1be-e87a-4458-baf3-ab403a4daa2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:41.457156 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c1be-e87a-4458-baf3-ab403a4daa2a service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:41.481348 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61eb0f-7696-4269-82be-01a0dc8ff09f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:41.482110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61eb0f-7696-4269-82be-01a0dc8ff09f service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:41.482746 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61eb0f-7696-4269-82be-01a0dc8ff09f service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:41.482746 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61eb0f-7696-4269-82be-01a0dc8ff09f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:41.512460 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61eb0f-7696-4269-82be-01a0dc8ff09f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:41.514284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61eb0f-7696-4269-82be-01a0dc8ff09f service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:41.844270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 705/1395] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:41.847928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fb4f2-e63f-4579-a698-d05a583feaf2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:41.847928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fb4f2-e63f-4579-a698-d05a583feaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:41.847928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fb4f2-e63f-4579-a698-d05a583feaf2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:41.847928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 691/1396] 10.222.0.30 () {54 vars in 912 bytes} [Sat Oct 15 13:03:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:03:41.854193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a0cd-d054-494a-bdc8-99d432d9beb4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:41.854193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a0cd-d054-494a-bdc8-99d432d9beb4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:41.854193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a0cd-d054-494a-bdc8-99d432d9beb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:41.854193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-cce9a0cd-d054-494a-bdc8-99d432d9beb4 None None] MFA Rules not processed for user `7a459d7bb6ea42ed93c5bd8eea8a37b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:41.854193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cce9a0cd-d054-494a-bdc8-99d432d9beb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:41.919021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 706/1397] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 13:03:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:41.946185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fb4f2-e63f-4579-a698-d05a583feaf2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:41.948223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a21fb4f2-e63f-4579-a698-d05a583feaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:41.980786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fb4f2-e63f-4579-a698-d05a583feaf2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:41.989465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50751080-b657-4f00-80d6-d76e27b28d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:41.993647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751080-b657-4f00-80d6-d76e27b28d60 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:41.993647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751080-b657-4f00-80d6-d76e27b28d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:41.993647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751080-b657-4f00-80d6-d76e27b28d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:42.008476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50751080-b657-4f00-80d6-d76e27b28d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:42.010016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50751080-b657-4f00-80d6-d76e27b28d60 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:42.027858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 692/1398] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:43.415404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a0cd-d054-494a-bdc8-99d432d9beb4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:43.417909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cce9a0cd-d054-494a-bdc8-99d432d9beb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:43.582965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:43.645626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a0cd-d054-494a-bdc8-99d432d9beb4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:43.660883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a92d-1566-448d-8b19-a7db0216e055 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18df1f401bb2494299eda9582c85a04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:43.661599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a92d-1566-448d-8b19-a7db0216e055 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:43.662021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a92d-1566-448d-8b19-a7db0216e055 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:43.662980 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a92d-1566-448d-8b19-a7db0216e055 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:43.669622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fc47a92d-1566-448d-8b19-a7db0216e055 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:43.703943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a92d-1566-448d-8b19-a7db0216e055 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:43.706887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a92d-1566-448d-8b19-a7db0216e055 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:43.738857 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 707/1399] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:45.230251 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51550b9-8693-483a-ae55-8daf71d6afdb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:45.230251 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51550b9-8693-483a-ae55-8daf71d6afdb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:45.230251 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51550b9-8693-483a-ae55-8daf71d6afdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:45.344457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c51550b9-8693-483a-ae55-8daf71d6afdb None None] MFA Rules not processed for user `51adf5d219a74f26a1e70cdf65c771f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:45.347685 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c51550b9-8693-483a-ae55-8daf71d6afdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:45.544299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 693/1400] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:45.560672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a92d-1566-448d-8b19-a7db0216e055 service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:45.584768 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a92d-1566-448d-8b19-a7db0216e055 service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:45.591759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d56f39-745a-4ae4-ac3d-44b24b6fbb09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:45.592760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d56f39-745a-4ae4-ac3d-44b24b6fbb09 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:45.593177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d56f39-745a-4ae4-ac3d-44b24b6fbb09 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:45.593703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d56f39-745a-4ae4-ac3d-44b24b6fbb09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:45.608629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d56f39-745a-4ae4-ac3d-44b24b6fbb09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:45.610819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d56f39-745a-4ae4-ac3d-44b24b6fbb09 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:45.647292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 708/1401] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:45.869033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb3dd-6733-41f6-8265-01135e82fbd0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:45.869372 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb3dd-6733-41f6-8265-01135e82fbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:45.869654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb3dd-6733-41f6-8265-01135e82fbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:45.895868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:45.981449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-af9eb3dd-6733-41f6-8265-01135e82fbd0 None None] MFA Rules not processed for user `f2b548d559b54575afd98739340ccf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:45.989859 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-af9eb3dd-6733-41f6-8265-01135e82fbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:46.179972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 694/1402] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:46.193341 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d56f39-745a-4ae4-ac3d-44b24b6fbb09 service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:46.214591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d56f39-745a-4ae4-ac3d-44b24b6fbb09 service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:46.222719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb1696-f4c7-435e-8e0f-e4c5b437a3f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:46.223572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1696-f4c7-435e-8e0f-e4c5b437a3f9 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:46.224043 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1696-f4c7-435e-8e0f-e4c5b437a3f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:46.224448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1696-f4c7-435e-8e0f-e4c5b437a3f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:46.237035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb1696-f4c7-435e-8e0f-e4c5b437a3f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:46.239295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb1696-f4c7-435e-8e0f-e4c5b437a3f9 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:46.251139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 709/1403] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:46.563865 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12629cc-115c-4f85-bd58-8dfd3b140084 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:46.565076 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12629cc-115c-4f85-bd58-8dfd3b140084 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:46.565599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12629cc-115c-4f85-bd58-8dfd3b140084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:46.630725 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d12629cc-115c-4f85-bd58-8dfd3b140084 None None] MFA Rules not processed for user `c36b6b9fbf194346bfc0a7e277c72eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:46.632430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d12629cc-115c-4f85-bd58-8dfd3b140084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:46.767963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 695/1404] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:46.779218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56961d7e-fb8b-4bdd-8a7f-9d7208ac06fc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:46.779737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56961d7e-fb8b-4bdd-8a7f-9d7208ac06fc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:46.779737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56961d7e-fb8b-4bdd-8a7f-9d7208ac06fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:46.805373 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-56961d7e-fb8b-4bdd-8a7f-9d7208ac06fc None None] MFA Rules not processed for user `c36b6b9fbf194346bfc0a7e277c72eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:46.806852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-56961d7e-fb8b-4bdd-8a7f-9d7208ac06fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:46.822127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 710/1405] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:46.833003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12629cc-115c-4f85-bd58-8dfd3b140084 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:46.851337 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12629cc-115c-4f85-bd58-8dfd3b140084 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:46.865529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b538738-64d5-48ab-8e06-e5ed8254fbc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:46.866225 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b538738-64d5-48ab-8e06-e5ed8254fbc0 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:46.866510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b538738-64d5-48ab-8e06-e5ed8254fbc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:46.866790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b538738-64d5-48ab-8e06-e5ed8254fbc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:46.886252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b538738-64d5-48ab-8e06-e5ed8254fbc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:46.895410 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b538738-64d5-48ab-8e06-e5ed8254fbc0 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:46.918078 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 696/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:46 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:47.840392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bb7da-067b-4e3a-9d5a-e53084172feb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:47.841398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bb7da-067b-4e3a-9d5a-e53084172feb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:47.841398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bb7da-067b-4e3a-9d5a-e53084172feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:47.881043 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d26bb7da-067b-4e3a-9d5a-e53084172feb None None] MFA Rules not processed for user `cfbb59c3a18a4bbaa752cbee63ae128d`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:47.887007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d26bb7da-067b-4e3a-9d5a-e53084172feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:47.930733 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 711/1407] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:48.449594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc66c7b-78f2-455b-a94d-d5b225d023b7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:48.450145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc66c7b-78f2-455b-a94d-d5b225d023b7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:48.450145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc66c7b-78f2-455b-a94d-d5b225d023b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:48.525604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3fc66c7b-78f2-455b-a94d-d5b225d023b7 None None] MFA Rules not processed for user `c36b6b9fbf194346bfc0a7e277c72eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:48.526873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3fc66c7b-78f2-455b-a94d-d5b225d023b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:48.560885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 697/1408] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:49.114339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16fe07-e582-49d1-9e00-cdd2b7104a4b None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:49.114825 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16fe07-e582-49d1-9e00-cdd2b7104a4b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:49.114943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16fe07-e582-49d1-9e00-cdd2b7104a4b None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:49.116551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 712/1409] 10.222.0.30 () {54 vars in 912 bytes} [Sat Oct 15 13:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:03:49.126505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404769c6-0dc5-4b4c-bbad-d1eda799a455 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:49.131484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404769c6-0dc5-4b4c-bbad-d1eda799a455 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:49.131484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404769c6-0dc5-4b4c-bbad-d1eda799a455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:49.234878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-404769c6-0dc5-4b4c-bbad-d1eda799a455 None None] MFA Rules not processed for user `7a459d7bb6ea42ed93c5bd8eea8a37b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:49.239288 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-404769c6-0dc5-4b4c-bbad-d1eda799a455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:49.257209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 698/1410] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 13:03:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:49.339395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16fe07-e582-49d1-9e00-cdd2b7104a4b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:49.341631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-de16fe07-e582-49d1-9e00-cdd2b7104a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:49.365020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16fe07-e582-49d1-9e00-cdd2b7104a4b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:49.382477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6408472-2567-4aa6-876e-e742b3eaec20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:49.383475 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408472-2567-4aa6-876e-e742b3eaec20 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:49.383953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408472-2567-4aa6-876e-e742b3eaec20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:49.384472 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408472-2567-4aa6-876e-e742b3eaec20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:49.396703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6408472-2567-4aa6-876e-e742b3eaec20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:49.399087 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6408472-2567-4aa6-876e-e742b3eaec20 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:49.413384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 713/1411] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:50.757664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29e449-0226-4513-b1cd-a029045892ef None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:50.758241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29e449-0226-4513-b1cd-a029045892ef None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:50.758241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29e449-0226-4513-b1cd-a029045892ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:50.793869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-af29e449-0226-4513-b1cd-a029045892ef None None] MFA Rules not processed for user `51adf5d219a74f26a1e70cdf65c771f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:50.795236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-af29e449-0226-4513-b1cd-a029045892ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:50.813397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 699/1412] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:52.053120 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad12eb5-71c3-4928-993e-953cc08c3eb9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:52.053697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad12eb5-71c3-4928-993e-953cc08c3eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:52.053697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad12eb5-71c3-4928-993e-953cc08c3eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:52.088315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dad12eb5-71c3-4928-993e-953cc08c3eb9 None None] MFA Rules not processed for user `f2b548d559b54575afd98739340ccf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:52.090708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dad12eb5-71c3-4928-993e-953cc08c3eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:52.102815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 714/1413] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:53.137321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29e449-0226-4513-b1cd-a029045892ef None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:53.324732 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:53.358712 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29e449-0226-4513-b1cd-a029045892ef None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:53.373098 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:53.376056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:53.376477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:53.379479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] PATH_INFO: `/v3/users/edc1b2c441034889a7625bd5f3f4c042` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:53.380956 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc1b2c441034889a7625bd5f3f4c042)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:53.383621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:53.597788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc1b2c441034889a7625bd5f3f4c042'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:53.599066 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc1b2c441034889a7625bd5f3f4c042'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:53.658195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc1b2c441034889a7625bd5f3f4c042'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:53.660299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc1b2c441034889a7625bd5f3f4c042'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:53.745854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77bb4a4b-0de1-4791-a122-8fa054147efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc1b2c441034889a7625bd5f3f4c042'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:53.797385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 700/1414] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:03:53 2022] DELETE /identity/v3/users/edc1b2c441034889a7625bd5f3f4c042 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:53.802519 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a2ebc-318c-4b15-ab46-1baea2f15481 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:53.803418 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a2ebc-318c-4b15-ab46-1baea2f15481 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:53.804218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a2ebc-318c-4b15-ab46-1baea2f15481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:53.849297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-249a2ebc-318c-4b15-ab46-1baea2f15481 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:53.853474 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-249a2ebc-318c-4b15-ab46-1baea2f15481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:53.902564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3948af-85be-4ae4-b5fb-a1834edf81c1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:53.903142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3948af-85be-4ae4-b5fb-a1834edf81c1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:53.903705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3948af-85be-4ae4-b5fb-a1834edf81c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:54.013019 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 715/1415] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:54.032731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3d3948af-85be-4ae4-b5fb-a1834edf81c1 None None] MFA Rules not processed for user `16c94590fafa48f58fd94526a913dd66`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:54.035733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3d3948af-85be-4ae4-b5fb-a1834edf81c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:54.051492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a2ebc-318c-4b15-ab46-1baea2f15481 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:54.053256 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-249a2ebc-318c-4b15-ab46-1baea2f15481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:54.272855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a2ebc-318c-4b15-ab46-1baea2f15481 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:54.277645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3bfee-0dcf-4b85-8e81-01cfcfaf9c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:54.278493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3bfee-0dcf-4b85-8e81-01cfcfaf9c9f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:54.278915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3bfee-0dcf-4b85-8e81-01cfcfaf9c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:54.279326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3bfee-0dcf-4b85-8e81-01cfcfaf9c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:54.280872 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 701/1416] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 13:03:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:54.306650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3bfee-0dcf-4b85-8e81-01cfcfaf9c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:54.309489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3bfee-0dcf-4b85-8e81-01cfcfaf9c9f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:54.339477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 716/1417] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:54.812303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3948af-85be-4ae4-b5fb-a1834edf81c1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:54.813993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3d3948af-85be-4ae4-b5fb-a1834edf81c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:54.931118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:55.093368 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3948af-85be-4ae4-b5fb-a1834edf81c1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:55.127938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ef4ec-027d-411d-bddd-75ab892d199f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b849f213606a4686bfb140fec34d512f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:55.128751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ef4ec-027d-411d-bddd-75ab892d199f service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:55.129661 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ef4ec-027d-411d-bddd-75ab892d199f service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:55.130972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ef4ec-027d-411d-bddd-75ab892d199f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:55.154349 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ef4ec-027d-411d-bddd-75ab892d199f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:55.159922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ef4ec-027d-411d-bddd-75ab892d199f service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:55.188652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 702/1418] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 13:03:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:03:57.094945 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98574f96-9b0c-4ab7-8510-abb75dbfc19e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:57.096153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98574f96-9b0c-4ab7-8510-abb75dbfc19e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:57.096153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98574f96-9b0c-4ab7-8510-abb75dbfc19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:57.175326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ef4ec-027d-411d-bddd-75ab892d199f service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:57.177399 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-108ef4ec-027d-411d-bddd-75ab892d199f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:57.227319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ef4ec-027d-411d-bddd-75ab892d199f service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:57.249777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-98574f96-9b0c-4ab7-8510-abb75dbfc19e None None] MFA Rules not processed for user `7a459d7bb6ea42ed93c5bd8eea8a37b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:57.251527 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:57.255254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-98574f96-9b0c-4ab7-8510-abb75dbfc19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:57.256551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:57.258509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:57.262011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] PATH_INFO: `/v3/projects/7f5821d5735840b499a3e954a2ad1319` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:57.267025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5821d5735840b499a3e954a2ad1319)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:57.274938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:57.328641 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 717/1419] 10.222.0.39 () {58 vars in 1015 bytes} [Sat Oct 15 13:03:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:57.451130 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98574f96-9b0c-4ab7-8510-abb75dbfc19e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:57.472314 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98574f96-9b0c-4ab7-8510-abb75dbfc19e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:57.484907 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061020ef-30d4-423a-b0b0-ae5dadb32387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:57.491790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061020ef-30d4-423a-b0b0-ae5dadb32387 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:57.492084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061020ef-30d4-423a-b0b0-ae5dadb32387 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:57.492367 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061020ef-30d4-423a-b0b0-ae5dadb32387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:57.529556 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061020ef-30d4-423a-b0b0-ae5dadb32387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:57.531054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061020ef-30d4-423a-b0b0-ae5dadb32387 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:57.566025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:57.590705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5821d5735840b499a3e954a2ad1319', 'request_id': 'req-45d289f4-73db-46ae-9835-435a4db9dde2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:57.591397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 718/1420] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:57.609469 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5821d5735840b499a3e954a2ad1319', 'request_id': 'req-45d289f4-73db-46ae-9835-435a4db9dde2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:57.660909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061020ef-30d4-423a-b0b0-ae5dadb32387 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:57.679223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061020ef-30d4-423a-b0b0-ae5dadb32387 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:57.721747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5821d5735840b499a3e954a2ad1319', 'request_id': 'req-45d289f4-73db-46ae-9835-435a4db9dde2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:57.752610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] The token cache is being invalidate because project 7f5821d5735840b499a3e954a2ad1319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:03:57.756689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:57.756689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:57.772488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 703/1421] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:03:57 2022] DELETE /identity/v3/projects/7f5821d5735840b499a3e954a2ad1319 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:03:57.773994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:57.791192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:57.902223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:57.910245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:57.913852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:57.913852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:57.913852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:58.006352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:58.012934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:58.063267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d289f4-73db-46ae-9835-435a4db9dde2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:58.075377 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 719/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:03:58.081452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:58.082175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:58.084001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:58.084418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] PATH_INFO: `/v3/users/0be1a13741fe42a2ab8845d0b452fac8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:58.087603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be1a13741fe42a2ab8845d0b452fac8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:58.089805 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:58.340120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:03:58.370436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be1a13741fe42a2ab8845d0b452fac8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:58.371352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be1a13741fe42a2ab8845d0b452fac8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:58.390407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be1a13741fe42a2ab8845d0b452fac8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:58.391180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be1a13741fe42a2ab8845d0b452fac8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:58.476346 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98a41f96-ee1f-4e8c-98d3-573a20a8a4e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be1a13741fe42a2ab8845d0b452fac8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:58.511109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 704/1423] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:03:57 2022] DELETE /identity/v3/users/0be1a13741fe42a2ab8845d0b452fac8 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:58.680429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:58.683240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:58.758547 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a868c-b7c0-4660-b011-2f0712c23c70 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:03:58.942437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:03:58.943107 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:58.943390 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:58.943667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] PATH_INFO: `/v3/users/c4ab77eb28d746ceb2824340614d2167` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:58.958168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ab77eb28d746ceb2824340614d2167)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:03:58.964247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:03:59.246814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ab77eb28d746ceb2824340614d2167'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:59.268639 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ab77eb28d746ceb2824340614d2167'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:59.273677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ab77eb28d746ceb2824340614d2167'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:59.278736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ab77eb28d746ceb2824340614d2167'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:59.391160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ab77eb28d746ceb2824340614d2167'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:03:59.491304 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 720/1424] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:03:58 2022] DELETE /identity/v3/users/c4ab77eb28d746ceb2824340614d2167 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:03:59.491510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabde804-b105-4b82-a153-e98166e7fed4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:03:59.491510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabde804-b105-4b82-a153-e98166e7fed4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:03:59.491510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabde804-b105-4b82-a153-e98166e7fed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:03:59.521388 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fabde804-b105-4b82-a153-e98166e7fed4 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:03:59.545599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fabde804-b105-4b82-a153-e98166e7fed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:03:59.796073 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 705/1425] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:03:59.989362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:03:59.991046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:00.074964 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:00.139144 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce45f0-cdaf-41cc-82cf-6919aa6acdad admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:00.143049 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb044a4b-cd61-4558-bd45-d685c83b6549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:00.143685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb044a4b-cd61-4558-bd45-d685c83b6549 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:00.144020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb044a4b-cd61-4558-bd45-d685c83b6549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:00.144302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb044a4b-cd61-4558-bd45-d685c83b6549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:00.154759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb044a4b-cd61-4558-bd45-d685c83b6549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:00.156230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb044a4b-cd61-4558-bd45-d685c83b6549 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:00.174088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 721/1426] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:01.167116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabde804-b105-4b82-a153-e98166e7fed4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:01.175007 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fabde804-b105-4b82-a153-e98166e7fed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:01.214950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabde804-b105-4b82-a153-e98166e7fed4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:01.223558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:01.225238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:01.225238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:01.230185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] PATH_INFO: `/v3/projects/7cb39bc6e8a74d90a7794a695af33cfd` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:01.246593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb39bc6e8a74d90a7794a695af33cfd)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:01.249145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:01.487730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb39bc6e8a74d90a7794a695af33cfd', 'request_id': 'req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:01.500037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb39bc6e8a74d90a7794a695af33cfd', 'request_id': 'req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:01.603393 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb39bc6e8a74d90a7794a695af33cfd', 'request_id': 'req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:01.630780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] The token cache is being invalidate because project 7cb39bc6e8a74d90a7794a695af33cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:01.631337 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:01.635494 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7ebcf0e8-e2d3-41b9-954f-6049d38a07f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:01.648995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 706/1427] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:01 2022] DELETE /identity/v3/projects/7cb39bc6e8a74d90a7794a695af33cfd => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:01.649342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abac65-caab-40e1-979a-ebf549a9d972 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:01.649342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abac65-caab-40e1-979a-ebf549a9d972 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:01.649342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abac65-caab-40e1-979a-ebf549a9d972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:01.692677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-05abac65-caab-40e1-979a-ebf549a9d972 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:01.694025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-05abac65-caab-40e1-979a-ebf549a9d972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:01.868688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 722/1428] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:01.872775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af121a-4f94-42eb-95b5-a152d4a229fd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:01.872775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af121a-4f94-42eb-95b5-a152d4a229fd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:01.872775 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af121a-4f94-42eb-95b5-a152d4a229fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:01.898256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-74af121a-4f94-42eb-95b5-a152d4a229fd None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:01.909414 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-74af121a-4f94-42eb-95b5-a152d4a229fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:01.916117 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abac65-caab-40e1-979a-ebf549a9d972 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:01.922185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 707/1429] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:01.937874 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abac65-caab-40e1-979a-ebf549a9d972 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:01.948723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd1916-085f-4c7a-a9ff-ff59a197a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:01.949407 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd1916-085f-4c7a-a9ff-ff59a197a537 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:01.951270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd1916-085f-4c7a-a9ff-ff59a197a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:01.951580 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd1916-085f-4c7a-a9ff-ff59a197a537 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:01.952279 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd1916-085f-4c7a-a9ff-ff59a197a537 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:01.952594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd1916-085f-4c7a-a9ff-ff59a197a537 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:01.972010 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd1916-085f-4c7a-a9ff-ff59a197a537 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:01.990323 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af121a-4f94-42eb-95b5-a152d4a229fd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:01.997120 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 723/1430] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:02.004297 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af121a-4f94-42eb-95b5-a152d4a229fd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:02.010426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b4de2-3630-4f88-be27-7d31187762d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:02.011891 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b4de2-3630-4f88-be27-7d31187762d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:02.012328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b4de2-3630-4f88-be27-7d31187762d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:02.012745 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b4de2-3630-4f88-be27-7d31187762d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:02.014139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b4de2-3630-4f88-be27-7d31187762d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:02.016850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b4de2-3630-4f88-be27-7d31187762d3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:02.570963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 708/1431] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:01 2022] POST /identity/v3/projects => generated 379 bytes in 581 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:02.575792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd1916-085f-4c7a-a9ff-ff59a197a537 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:02.601852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd1916-085f-4c7a-a9ff-ff59a197a537 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:02.773653 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96509f95-76b5-4835-a5d5-28338011494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:02.788788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96509f95-76b5-4835-a5d5-28338011494a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:02.788788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96509f95-76b5-4835-a5d5-28338011494a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:02.788967 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96509f95-76b5-4835-a5d5-28338011494a admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:02.789632 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96509f95-76b5-4835-a5d5-28338011494a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:02.791600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96509f95-76b5-4835-a5d5-28338011494a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:02.887549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b4de2-3630-4f88-be27-7d31187762d3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:02.890886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-834b4de2-3630-4f88-be27-7d31187762d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:02.921040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b4de2-3630-4f88-be27-7d31187762d3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:02.922951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 724/1432] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:02 2022] POST /identity/v3/users => generated 290 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:02.924281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96509f95-76b5-4835-a5d5-28338011494a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:02.947701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:02.950405 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:02.952547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:02.953037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] PATH_INFO: `/v3/projects/631e6b22fa1b4166a4371776687fffe6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:02.956360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96509f95-76b5-4835-a5d5-28338011494a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:02.958027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=631e6b22fa1b4166a4371776687fffe6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:02.962528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:02.974774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:02.984193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a3055-5f49-473c-943f-68a92d033423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:02.985056 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a3055-5f49-473c-943f-68a92d033423 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:02.985643 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a3055-5f49-473c-943f-68a92d033423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:02.986671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a3055-5f49-473c-943f-68a92d033423 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:02.987495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a3055-5f49-473c-943f-68a92d033423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:02.989606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a3055-5f49-473c-943f-68a92d033423 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:03.072605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 725/1433] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:03.075708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a3055-5f49-473c-943f-68a92d033423 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:03.138411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a3055-5f49-473c-943f-68a92d033423 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:03.159896 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bba06b-887f-49f9-a0b6-797b8a1cc198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:03.162487 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bba06b-887f-49f9-a0b6-797b8a1cc198 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:03.162892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bba06b-887f-49f9-a0b6-797b8a1cc198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:03.163418 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bba06b-887f-49f9-a0b6-797b8a1cc198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:03.181716 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bba06b-887f-49f9-a0b6-797b8a1cc198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:03.190144 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bba06b-887f-49f9-a0b6-797b8a1cc198 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:03.257785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 726/1434] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:03.269145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342f137-2015-4d38-92e2-a07ba67158a5 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:03.269145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342f137-2015-4d38-92e2-a07ba67158a5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:03.269145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342f137-2015-4d38-92e2-a07ba67158a5 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:03.281629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631e6b22fa1b4166a4371776687fffe6', 'request_id': 'req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:03.282007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 727/1435] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 13:04:03 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:04:03.286548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342f137-2015-4d38-92e2-a07ba67158a5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:03.313633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631e6b22fa1b4166a4371776687fffe6', 'request_id': 'req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:03.394388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:03.397930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631e6b22fa1b4166a4371776687fffe6', 'request_id': 'req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:03.424064 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] The token cache is being invalidate because project 631e6b22fa1b4166a4371776687fffe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:03.424417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:03.430104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:03.436210 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 709/1436] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:02 2022] DELETE /identity/v3/projects/631e6b22fa1b4166a4371776687fffe6 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:03.440862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:03.444661 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:03.483191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342f137-2015-4d38-92e2-a07ba67158a5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:03.489879 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e68c66-f77e-41e2-b686-b0b15270e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:03.490762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e68c66-f77e-41e2-b686-b0b15270e64b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:03.491056 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e68c66-f77e-41e2-b686-b0b15270e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:03.491460 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e68c66-f77e-41e2-b686-b0b15270e64b admin admin] PATH_INFO: `/v3/projects/7b5abd4b70eb4b18af00352a0f33c924/users/a441584aa5a6409e86b1f84f8bcb28a4/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:03.519738 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e68c66-f77e-41e2-b686-b0b15270e64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5abd4b70eb4b18af00352a0f33c924, user_id=a441584aa5a6409e86b1f84f8bcb28a4, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:03.523956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e68c66-f77e-41e2-b686-b0b15270e64b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:03.584221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 728/1437] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:03 2022] PUT /identity/v3/projects/7b5abd4b70eb4b18af00352a0f33c924/users/a441584aa5a6409e86b1f84f8bcb28a4/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:03.596121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c1f2-920b-4710-aee3-5edcc58da082 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:03.596454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c1f2-920b-4710-aee3-5edcc58da082 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:03.596792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c1f2-920b-4710-aee3-5edcc58da082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:03.651156 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3796c1f2-920b-4710-aee3-5edcc58da082 None None] MFA Rules not processed for user `a441584aa5a6409e86b1f84f8bcb28a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:03.654100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3796c1f2-920b-4710-aee3-5edcc58da082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:03.722801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3130f-2aec-4f7c-a33c-37c08d6d5a8a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:03.885345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 729/1438] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:03.888228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5539d8a-4221-4adf-9348-e1904a9db710 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:03.896832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5539d8a-4221-4adf-9348-e1904a9db710 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:03.897356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5539d8a-4221-4adf-9348-e1904a9db710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:03.979444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a5539d8a-4221-4adf-9348-e1904a9db710 None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:03.989047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a5539d8a-4221-4adf-9348-e1904a9db710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:03.995547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:03.996198 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:03.996818 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:03.997108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] PATH_INFO: `/v3/users/9fcc6fabb2a04de7bb3b93cfc23013e7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:04.007847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fcc6fabb2a04de7bb3b93cfc23013e7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:04.018030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:04.136110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 730/1439] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 13:04:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:04.137226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5539d8a-4221-4adf-9348-e1904a9db710 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:04.138568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a5539d8a-4221-4adf-9348-e1904a9db710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:04.156364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5539d8a-4221-4adf-9348-e1904a9db710 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:04.179852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06e1f3-f69f-4daa-9037-9889a80c3b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:04.180576 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06e1f3-f69f-4daa-9037-9889a80c3b20 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:04.181211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06e1f3-f69f-4daa-9037-9889a80c3b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:04.181572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06e1f3-f69f-4daa-9037-9889a80c3b20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:04.182297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06e1f3-f69f-4daa-9037-9889a80c3b20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:04.185552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06e1f3-f69f-4daa-9037-9889a80c3b20 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:04.318596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 731/1440] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:04 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:04.322104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:04.324129 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06e1f3-f69f-4daa-9037-9889a80c3b20 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:04.336442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:04.343611 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fcc6fabb2a04de7bb3b93cfc23013e7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:04.348413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fcc6fabb2a04de7bb3b93cfc23013e7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:04.382315 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fcc6fabb2a04de7bb3b93cfc23013e7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:04.382315 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fcc6fabb2a04de7bb3b93cfc23013e7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:04.494837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fcc6fabb2a04de7bb3b93cfc23013e7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:04.508745 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06e1f3-f69f-4daa-9037-9889a80c3b20 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:04.510445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d8050-33a2-4761-bb81-55c134bbf19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:04.511210 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d8050-33a2-4761-bb81-55c134bbf19f admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:04.511562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d8050-33a2-4761-bb81-55c134bbf19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:04.511911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d8050-33a2-4761-bb81-55c134bbf19f admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:04.512628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d8050-33a2-4761-bb81-55c134bbf19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:04.514826 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d8050-33a2-4761-bb81-55c134bbf19f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:04.582864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 710/1441] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:03 2022] DELETE /identity/v3/users/9fcc6fabb2a04de7bb3b93cfc23013e7 => generated 0 bytes in 1142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:04.611802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 732/1442] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:04 2022] POST /identity/v3/users => generated 290 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:04.613171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:04.616174 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:04.760188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385d738-8d53-4a45-b051-f716d9c7ed1a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:04.771150 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac205bf-8ab2-48f1-940f-8e6b7fb9239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:04.772999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac205bf-8ab2-48f1-940f-8e6b7fb9239f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:04.773420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac205bf-8ab2-48f1-940f-8e6b7fb9239f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:04.774035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac205bf-8ab2-48f1-940f-8e6b7fb9239f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:04.775275 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac205bf-8ab2-48f1-940f-8e6b7fb9239f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:04.777661 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac205bf-8ab2-48f1-940f-8e6b7fb9239f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:04.791964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 711/1443] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:04.796747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d8050-33a2-4761-bb81-55c134bbf19f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:04.817135 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d8050-33a2-4761-bb81-55c134bbf19f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:04.820288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b70d8-033c-4cad-9b18-cb4022db48c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:04.820990 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b70d8-033c-4cad-9b18-cb4022db48c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:04.821376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b70d8-033c-4cad-9b18-cb4022db48c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:04.821723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b70d8-033c-4cad-9b18-cb4022db48c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:04.822890 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b70d8-033c-4cad-9b18-cb4022db48c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:04.826236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b70d8-033c-4cad-9b18-cb4022db48c7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:04.844705 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 733/1444] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:04.851483 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac205bf-8ab2-48f1-940f-8e6b7fb9239f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:04.870193 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac205bf-8ab2-48f1-940f-8e6b7fb9239f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:04.877926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b41ad4-1269-4a35-b262-ae98d5278216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:04.878737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b41ad4-1269-4a35-b262-ae98d5278216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:04.879204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b41ad4-1269-4a35-b262-ae98d5278216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:04.879543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b41ad4-1269-4a35-b262-ae98d5278216 admin admin] PATH_INFO: `/v3/projects/aa6224ee09ae4e868c23a0da09738468/users/74169ab9e3c04f90ad0594c2c4e0a7bb/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:04.910986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b41ad4-1269-4a35-b262-ae98d5278216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6224ee09ae4e868c23a0da09738468, user_id=74169ab9e3c04f90ad0594c2c4e0a7bb, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:04.914552 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b41ad4-1269-4a35-b262-ae98d5278216 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:04.993544 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 712/1445] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:04 2022] PUT /identity/v3/projects/aa6224ee09ae4e868c23a0da09738468/users/74169ab9e3c04f90ad0594c2c4e0a7bb/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:05.014213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073864ec-dfa0-4675-80a1-167375b61ed7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:05.014213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073864ec-dfa0-4675-80a1-167375b61ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:05.014213 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073864ec-dfa0-4675-80a1-167375b61ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:05.066238 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-073864ec-dfa0-4675-80a1-167375b61ed7 None None] MFA Rules not processed for user `74169ab9e3c04f90ad0594c2c4e0a7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:05.067867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-073864ec-dfa0-4675-80a1-167375b61ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:05.228634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 734/1446] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:05.234929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b41ad4-1269-4a35-b262-ae98d5278216 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:05.244543 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073864ec-dfa0-4675-80a1-167375b61ed7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:05.247872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-073864ec-dfa0-4675-80a1-167375b61ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:05.374182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073864ec-dfa0-4675-80a1-167375b61ed7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:05.374689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b41ad4-1269-4a35-b262-ae98d5278216 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:05.380785 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cdd4a-e7f3-4015-92cf-c03fdcfd641d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:05.382901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cdd4a-e7f3-4015-92cf-c03fdcfd641d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:05.383291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cdd4a-e7f3-4015-92cf-c03fdcfd641d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:05.383615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cdd4a-e7f3-4015-92cf-c03fdcfd641d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:05.384418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cdd4a-e7f3-4015-92cf-c03fdcfd641d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:05.384625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a2d3d-0d00-4fc5-97b9-60734146b94a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:05.385480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a2d3d-0d00-4fc5-97b9-60734146b94a service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:05.385957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a2d3d-0d00-4fc5-97b9-60734146b94a service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:05.386401 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a2d3d-0d00-4fc5-97b9-60734146b94a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:05.388495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cdd4a-e7f3-4015-92cf-c03fdcfd641d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:05.389953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-957a2d3d-0d00-4fc5-97b9-60734146b94a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:05.410476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a2d3d-0d00-4fc5-97b9-60734146b94a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:05.415541 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a2d3d-0d00-4fc5-97b9-60734146b94a service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:05.448392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 735/1447] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:05.497771 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 713/1448] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:05 2022] POST /identity/v3/projects => generated 379 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:05.499097 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a2d3d-0d00-4fc5-97b9-60734146b94a service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:05.519361 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a2d3d-0d00-4fc5-97b9-60734146b94a service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:05.728153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c7f97-67d6-4576-aeb2-f1299d266c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:05.729034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c7f97-67d6-4576-aeb2-f1299d266c1c admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:05.729359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c7f97-67d6-4576-aeb2-f1299d266c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:05.729777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c7f97-67d6-4576-aeb2-f1299d266c1c admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:05.730616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c7f97-67d6-4576-aeb2-f1299d266c1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:05.733528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c7f97-67d6-4576-aeb2-f1299d266c1c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:05.849048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cdd4a-e7f3-4015-92cf-c03fdcfd641d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:05.849664 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 736/1449] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:05 2022] POST /identity/v3/users => generated 296 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:05.874878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cdd4a-e7f3-4015-92cf-c03fdcfd641d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:05.903751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7a57c-fc61-413e-9f07-f79e1499f994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:05.906270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7a57c-fc61-413e-9f07-f79e1499f994 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:05.906826 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7a57c-fc61-413e-9f07-f79e1499f994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:05.907515 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7a57c-fc61-413e-9f07-f79e1499f994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:05.908739 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7a57c-fc61-413e-9f07-f79e1499f994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:05.911543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7a57c-fc61-413e-9f07-f79e1499f994 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:05.938627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 714/1450] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:05.943966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c7f97-67d6-4576-aeb2-f1299d266c1c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:05.955991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c7f97-67d6-4576-aeb2-f1299d266c1c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:05.976021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c03ad-cdce-4593-9eec-8f61ff9162f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:05.989291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c03ad-cdce-4593-9eec-8f61ff9162f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:05.989291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c03ad-cdce-4593-9eec-8f61ff9162f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:05.989291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:05.993178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c03ad-cdce-4593-9eec-8f61ff9162f0 admin admin] PATH_INFO: `/v3/projects/1ae15a69bb48499097c9b828452c1e0f/users/c9950eb4b476471e8dae0aae10a1cbac/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:06.033647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c03ad-cdce-4593-9eec-8f61ff9162f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae15a69bb48499097c9b828452c1e0f, user_id=c9950eb4b476471e8dae0aae10a1cbac, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:06.040522 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c03ad-cdce-4593-9eec-8f61ff9162f0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:06.144260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 737/1451] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:05 2022] PUT /identity/v3/projects/1ae15a69bb48499097c9b828452c1e0f/users/c9950eb4b476471e8dae0aae10a1cbac/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:06.150685 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7a57c-fc61-413e-9f07-f79e1499f994 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:06.178677 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7a57c-fc61-413e-9f07-f79e1499f994 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:06.342319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493bc77-a30a-48dd-8f6d-dd9b9c881b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:06.343802 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493bc77-a30a-48dd-8f6d-dd9b9c881b30 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:06.343802 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493bc77-a30a-48dd-8f6d-dd9b9c881b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:06.343802 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493bc77-a30a-48dd-8f6d-dd9b9c881b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:06.353499 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493bc77-a30a-48dd-8f6d-dd9b9c881b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:06.356205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493bc77-a30a-48dd-8f6d-dd9b9c881b30 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:06.395860 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c03ad-cdce-4593-9eec-8f61ff9162f0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:06.396472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 715/1452] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:06.409717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c03ad-cdce-4593-9eec-8f61ff9162f0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:06.422716 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2236b-35be-4811-b09a-e0120f521617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:06.424773 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2236b-35be-4811-b09a-e0120f521617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:06.425340 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2236b-35be-4811-b09a-e0120f521617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:06.426052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2236b-35be-4811-b09a-e0120f521617 admin admin] PATH_INFO: `/v3/projects/1ae15a69bb48499097c9b828452c1e0f/users/c9950eb4b476471e8dae0aae10a1cbac/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:06.430695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2236b-35be-4811-b09a-e0120f521617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae15a69bb48499097c9b828452c1e0f, user_id=c9950eb4b476471e8dae0aae10a1cbac, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:06.435268 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2236b-35be-4811-b09a-e0120f521617 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:06.497559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f0084-04f3-4390-805e-afdc2aebdb6c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:06.497559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f0084-04f3-4390-805e-afdc2aebdb6c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:06.497559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f0084-04f3-4390-805e-afdc2aebdb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:06.497891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 738/1453] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:06 2022] PUT /identity/v3/projects/1ae15a69bb48499097c9b828452c1e0f/users/c9950eb4b476471e8dae0aae10a1cbac/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:06.503849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:06.555557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2c8f0084-04f3-4390-805e-afdc2aebdb6c None None] MFA Rules not processed for user `c9950eb4b476471e8dae0aae10a1cbac`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:06.556449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2c8f0084-04f3-4390-805e-afdc2aebdb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:06.745477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 716/1454] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:06.800754 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2236b-35be-4811-b09a-e0120f521617 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:06.805170 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bbe2236b-35be-4811-b09a-e0120f521617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:07.014389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2236b-35be-4811-b09a-e0120f521617 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:07.024210 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542ba2e-605f-44c8-aa38-7e80484fde3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:07.028944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542ba2e-605f-44c8-aa38-7e80484fde3e service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:07.028944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542ba2e-605f-44c8-aa38-7e80484fde3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:07.028944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542ba2e-605f-44c8-aa38-7e80484fde3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:07.060771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542ba2e-605f-44c8-aa38-7e80484fde3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:07.063262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542ba2e-605f-44c8-aa38-7e80484fde3e service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:07.088226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 739/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:07.292963 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f0084-04f3-4390-805e-afdc2aebdb6c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:07.306000 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f0084-04f3-4390-805e-afdc2aebdb6c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:07.320489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834add7f-b35b-4c72-9a23-6c50558da0f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:07.321251 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834add7f-b35b-4c72-9a23-6c50558da0f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:07.321539 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834add7f-b35b-4c72-9a23-6c50558da0f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:07.323033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834add7f-b35b-4c72-9a23-6c50558da0f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:07.327464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-834add7f-b35b-4c72-9a23-6c50558da0f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:07.453051 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1edcf-ea92-487e-abc0-f9f7a13a6dd7 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:07.453489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1edcf-ea92-487e-abc0-f9f7a13a6dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:07.453896 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1edcf-ea92-487e-abc0-f9f7a13a6dd7 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:07.455713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 740/1456] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 13:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:04:07.489253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834add7f-b35b-4c72-9a23-6c50558da0f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:07.492171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834add7f-b35b-4c72-9a23-6c50558da0f1 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:07.504896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 717/1457] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:07.507136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1edcf-ea92-487e-abc0-f9f7a13a6dd7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:07.513231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-16f1edcf-ea92-487e-abc0-f9f7a13a6dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:07.525908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c65d19-0af8-4132-b15b-4d920100f6e8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:07.525908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c65d19-0af8-4132-b15b-4d920100f6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:07.525908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c65d19-0af8-4132-b15b-4d920100f6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:07.611912 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-26c65d19-0af8-4132-b15b-4d920100f6e8 None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:07.613327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-26c65d19-0af8-4132-b15b-4d920100f6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:07.736162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1edcf-ea92-487e-abc0-f9f7a13a6dd7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:07.741521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:07.742260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:07.742542 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:07.742957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] PATH_INFO: `/v3/projects/02200fe4fa3944ac8f06679c7dd4b190` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:07.744449 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02200fe4fa3944ac8f06679c7dd4b190)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:07.746453 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:07.836849 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:07.857803 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 718/1458] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 13:04:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:07.920837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02200fe4fa3944ac8f06679c7dd4b190', 'request_id': 'req-89022b38-7ada-4106-aa67-2da94bd803b7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:07.996278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02200fe4fa3944ac8f06679c7dd4b190', 'request_id': 'req-89022b38-7ada-4106-aa67-2da94bd803b7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:08.007146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02200fe4fa3944ac8f06679c7dd4b190', 'request_id': 'req-89022b38-7ada-4106-aa67-2da94bd803b7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:08.046853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] The token cache is being invalidate because project 02200fe4fa3944ac8f06679c7dd4b190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:08.047149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:08.049159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-89022b38-7ada-4106-aa67-2da94bd803b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:08.055367 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 741/1459] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:07 2022] DELETE /identity/v3/projects/02200fe4fa3944ac8f06679c7dd4b190 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:08.061771 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff7e5e-46ca-4ecd-b618-70df958efb81 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:08.062168 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff7e5e-46ca-4ecd-b618-70df958efb81 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:08.062966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff7e5e-46ca-4ecd-b618-70df958efb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:08.124400 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-28ff7e5e-46ca-4ecd-b618-70df958efb81 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:08.127074 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-28ff7e5e-46ca-4ecd-b618-70df958efb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:08.227525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 719/1460] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:08.238753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaca8a-bb17-4785-b578-016786cb93f4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:08.239319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaca8a-bb17-4785-b578-016786cb93f4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:08.241505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaca8a-bb17-4785-b578-016786cb93f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:08.330245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-56aaca8a-bb17-4785-b578-016786cb93f4 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:08.330245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-56aaca8a-bb17-4785-b578-016786cb93f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:08.355186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 742/1461] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:08.356783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff7e5e-46ca-4ecd-b618-70df958efb81 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:08.377688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff7e5e-46ca-4ecd-b618-70df958efb81 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:08.388602 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd045210-803b-47d8-b0f6-01f9817e8170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:08.390768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd045210-803b-47d8-b0f6-01f9817e8170 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:08.391086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd045210-803b-47d8-b0f6-01f9817e8170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:08.391369 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd045210-803b-47d8-b0f6-01f9817e8170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:08.393537 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd045210-803b-47d8-b0f6-01f9817e8170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:08.401673 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd045210-803b-47d8-b0f6-01f9817e8170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:08.403363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd045210-803b-47d8-b0f6-01f9817e8170 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:08.440420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 720/1462] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:08.445165 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aaca8a-bb17-4785-b578-016786cb93f4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:08.465090 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aaca8a-bb17-4785-b578-016786cb93f4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:08.471614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93c2b8-06e4-4a23-809a-229081851273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:08.472820 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93c2b8-06e4-4a23-809a-229081851273 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:08.473330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93c2b8-06e4-4a23-809a-229081851273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:08.474027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93c2b8-06e4-4a23-809a-229081851273 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:08.474917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93c2b8-06e4-4a23-809a-229081851273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:08.477449 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93c2b8-06e4-4a23-809a-229081851273 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:08.614282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 743/1463] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:08 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:08.618295 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd045210-803b-47d8-b0f6-01f9817e8170 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:08.638905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd045210-803b-47d8-b0f6-01f9817e8170 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:08.792734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da456f3-b824-410c-9659-e9900d33b06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:08.793400 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da456f3-b824-410c-9659-e9900d33b06b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:08.793683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da456f3-b824-410c-9659-e9900d33b06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:08.793961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da456f3-b824-410c-9659-e9900d33b06b admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:08.794601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da456f3-b824-410c-9659-e9900d33b06b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:08.797030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da456f3-b824-410c-9659-e9900d33b06b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:08.956375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 721/1464] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:08 2022] POST /identity/v3/users => generated 283 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:08.956554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93c2b8-06e4-4a23-809a-229081851273 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:08.958240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-eb93c2b8-06e4-4a23-809a-229081851273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:08.964613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da456f3-b824-410c-9659-e9900d33b06b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:08.983974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da456f3-b824-410c-9659-e9900d33b06b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:09.002952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73139203-036b-4835-9c5e-c8fe58891561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:09.003476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73139203-036b-4835-9c5e-c8fe58891561 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:09.003760 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73139203-036b-4835-9c5e-c8fe58891561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:09.004036 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73139203-036b-4835-9c5e-c8fe58891561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:09.004697 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73139203-036b-4835-9c5e-c8fe58891561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:09.006334 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73139203-036b-4835-9c5e-c8fe58891561 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:09.024830 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:09.047881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 722/1465] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:09.052736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73139203-036b-4835-9c5e-c8fe58891561 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:09.065834 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73139203-036b-4835-9c5e-c8fe58891561 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:09.076966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f362af-eb4d-4c05-b48b-b81b7861bd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:09.077659 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f362af-eb4d-4c05-b48b-b81b7861bd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:09.078000 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f362af-eb4d-4c05-b48b-b81b7861bd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:09.078276 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f362af-eb4d-4c05-b48b-b81b7861bd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:09.078926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f362af-eb4d-4c05-b48b-b81b7861bd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:09.082182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f362af-eb4d-4c05-b48b-b81b7861bd04 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:09.096022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 723/1466] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:09.102388 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f362af-eb4d-4c05-b48b-b81b7861bd04 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:09.138636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f362af-eb4d-4c05-b48b-b81b7861bd04 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:09.147354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93c2b8-06e4-4a23-809a-229081851273 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:09.153895 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2fc3f-6e47-4aa9-8176-43d308097eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:09.156277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2fc3f-6e47-4aa9-8176-43d308097eaf service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:09.156277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2fc3f-6e47-4aa9-8176-43d308097eaf service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:09.156552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2fc3f-6e47-4aa9-8176-43d308097eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:09.159191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917c12-a01a-4f8d-89b1-513b17c61def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:09.159330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-39b2fc3f-6e47-4aa9-8176-43d308097eaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:09.164710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917c12-a01a-4f8d-89b1-513b17c61def admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:09.164710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917c12-a01a-4f8d-89b1-513b17c61def admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:09.164710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917c12-a01a-4f8d-89b1-513b17c61def admin admin] PATH_INFO: `/v3/projects/30ffd49709c54f678e177548ffba085d/users/c553103d2ec44e37a361aad3559ce6df/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:09.175908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2fc3f-6e47-4aa9-8176-43d308097eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:09.178469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2fc3f-6e47-4aa9-8176-43d308097eaf service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:09.193416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 744/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:09.232763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56917c12-a01a-4f8d-89b1-513b17c61def admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ffd49709c54f678e177548ffba085d, user_id=c553103d2ec44e37a361aad3559ce6df, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:09.236949 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56917c12-a01a-4f8d-89b1-513b17c61def admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:09.336607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 724/1468] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:09 2022] PUT /identity/v3/projects/30ffd49709c54f678e177548ffba085d/users/c553103d2ec44e37a361aad3559ce6df/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:09.349231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9b1b8-40e4-4a71-a27a-7963301c3556 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:09.349814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9b1b8-40e4-4a71-a27a-7963301c3556 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:09.349904 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9b1b8-40e4-4a71-a27a-7963301c3556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:09.404123 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bda9b1b8-40e4-4a71-a27a-7963301c3556 None None] MFA Rules not processed for user `c553103d2ec44e37a361aad3559ce6df`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:09.408466 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bda9b1b8-40e4-4a71-a27a-7963301c3556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:09.460459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:09.556577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:09.571989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 745/1469] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:09.576535 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917c12-a01a-4f8d-89b1-513b17c61def admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:09.752442 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917c12-a01a-4f8d-89b1-513b17c61def admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:09.763862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc897b-1c7f-4c92-8d3f-41ca2bd9e272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:09.764789 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc897b-1c7f-4c92-8d3f-41ca2bd9e272 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:09.765460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc897b-1c7f-4c92-8d3f-41ca2bd9e272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:09.765987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc897b-1c7f-4c92-8d3f-41ca2bd9e272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:09.767710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc897b-1c7f-4c92-8d3f-41ca2bd9e272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:09.779278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc897b-1c7f-4c92-8d3f-41ca2bd9e272 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:09.912540 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 725/1470] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:09 2022] POST /identity/v3/projects => generated 363 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:09.916947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9b1b8-40e4-4a71-a27a-7963301c3556 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:09.930484 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9b1b8-40e4-4a71-a27a-7963301c3556 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:10.013862 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eade6ef-db25-4566-8b46-6c8502463191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:10.014993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eade6ef-db25-4566-8b46-6c8502463191 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:10.015416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eade6ef-db25-4566-8b46-6c8502463191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:10.015830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eade6ef-db25-4566-8b46-6c8502463191 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:10.016672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eade6ef-db25-4566-8b46-6c8502463191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:10.019249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eade6ef-db25-4566-8b46-6c8502463191 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:10.107875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 746/1471] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:09 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:10.112481 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc897b-1c7f-4c92-8d3f-41ca2bd9e272 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:10.124714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc897b-1c7f-4c92-8d3f-41ca2bd9e272 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:10.130638 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c8ab9-ab13-4bff-8e55-f061e07980dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:10.131502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c8ab9-ab13-4bff-8e55-f061e07980dd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:10.131852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c8ab9-ab13-4bff-8e55-f061e07980dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:10.132331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c8ab9-ab13-4bff-8e55-f061e07980dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:10.133171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c8ab9-ab13-4bff-8e55-f061e07980dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:10.134936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c8ab9-ab13-4bff-8e55-f061e07980dd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:10.158627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eade6ef-db25-4566-8b46-6c8502463191 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:10.161693 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 726/1472] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:10.170408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eade6ef-db25-4566-8b46-6c8502463191 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:10.179069 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a63d7-4924-4650-8477-302c2991db04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:10.179977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a63d7-4924-4650-8477-302c2991db04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:10.180428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a63d7-4924-4650-8477-302c2991db04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:10.180866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a63d7-4924-4650-8477-302c2991db04 admin admin] PATH_INFO: `/v3/projects/b58ec448cefd4a20ad29e4864c011449/users/f3e7971a94804ae289e3f33fb6a93e26/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:10.202678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a63d7-4924-4650-8477-302c2991db04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58ec448cefd4a20ad29e4864c011449, user_id=f3e7971a94804ae289e3f33fb6a93e26, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:10.206004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a63d7-4924-4650-8477-302c2991db04 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:10.289476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 747/1473] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:10 2022] PUT /identity/v3/projects/b58ec448cefd4a20ad29e4864c011449/users/f3e7971a94804ae289e3f33fb6a93e26/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:10.290538 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c8ab9-ab13-4bff-8e55-f061e07980dd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:10.303253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c8ab9-ab13-4bff-8e55-f061e07980dd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:10.374498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de75c9-c2ad-4404-b949-cfc3f6213c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:10.375108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de75c9-c2ad-4404-b949-cfc3f6213c23 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:10.375392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de75c9-c2ad-4404-b949-cfc3f6213c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:10.375669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de75c9-c2ad-4404-b949-cfc3f6213c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:10.376897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de75c9-c2ad-4404-b949-cfc3f6213c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:10.378564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de75c9-c2ad-4404-b949-cfc3f6213c23 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:10.397530 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 727/1474] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:10.401572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a63d7-4924-4650-8477-302c2991db04 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:10.419466 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a63d7-4924-4650-8477-302c2991db04 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:10.425183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00f029-89e8-4c04-8e22-105f8852e9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:10.425953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00f029-89e8-4c04-8e22-105f8852e9e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:10.426253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00f029-89e8-4c04-8e22-105f8852e9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:10.426817 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00f029-89e8-4c04-8e22-105f8852e9e9 admin admin] PATH_INFO: `/v3/projects/b58ec448cefd4a20ad29e4864c011449/users/f3e7971a94804ae289e3f33fb6a93e26/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:10.428869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00f029-89e8-4c04-8e22-105f8852e9e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58ec448cefd4a20ad29e4864c011449, user_id=f3e7971a94804ae289e3f33fb6a93e26, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:10.431818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00f029-89e8-4c04-8e22-105f8852e9e9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:10.444440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 748/1475] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:10 2022] PUT /identity/v3/projects/b58ec448cefd4a20ad29e4864c011449/users/f3e7971a94804ae289e3f33fb6a93e26/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:10.456768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0c026-24cb-4c85-82ed-913938e34835 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:10.457303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0c026-24cb-4c85-82ed-913938e34835 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:10.457706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0c026-24cb-4c85-82ed-913938e34835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:10.499683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-82e0c026-24cb-4c85-82ed-913938e34835 None None] MFA Rules not processed for user `f3e7971a94804ae289e3f33fb6a93e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:10.499683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-82e0c026-24cb-4c85-82ed-913938e34835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:10.572782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:10.624792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 728/1476] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:10.643552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00f029-89e8-4c04-8e22-105f8852e9e9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:10.645237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2b00f029-89e8-4c04-8e22-105f8852e9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:10.720540 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00f029-89e8-4c04-8e22-105f8852e9e9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:10.724456 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0564f-bb03-4723-b07e-cce5a9c65adc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:10.725175 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0564f-bb03-4723-b07e-cce5a9c65adc service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:10.725680 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0564f-bb03-4723-b07e-cce5a9c65adc service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:10.725967 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0564f-bb03-4723-b07e-cce5a9c65adc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:10.736449 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0564f-bb03-4723-b07e-cce5a9c65adc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:10.738738 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0564f-bb03-4723-b07e-cce5a9c65adc service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:10.758272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 749/1477] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:11.030165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0c026-24cb-4c85-82ed-913938e34835 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:11.031871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-82e0c026-24cb-4c85-82ed-913938e34835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:11.049236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0c026-24cb-4c85-82ed-913938e34835 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:11.056146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e435a-9010-4839-b7fa-f40d845cf16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:11.057543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e435a-9010-4839-b7fa-f40d845cf16b service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:11.057978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e435a-9010-4839-b7fa-f40d845cf16b service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:11.058428 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e435a-9010-4839-b7fa-f40d845cf16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:11.152402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e435a-9010-4839-b7fa-f40d845cf16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:11.154693 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e435a-9010-4839-b7fa-f40d845cf16b service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:11.166022 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 729/1478] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:12.413280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0564f-bb03-4723-b07e-cce5a9c65adc service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:12.414744 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b8c0564f-bb03-4723-b07e-cce5a9c65adc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:12.451538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:12.507220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0564f-bb03-4723-b07e-cce5a9c65adc service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:12.512345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:12.512986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:12.513344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:12.513625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] PATH_INFO: `/v3/users/5196272eaf0846a186088131daa36cc9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:12.515468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5196272eaf0846a186088131daa36cc9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:12.517583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:12.698556 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5196272eaf0846a186088131daa36cc9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:12.711558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5196272eaf0846a186088131daa36cc9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:12.711558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5196272eaf0846a186088131daa36cc9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:12.711558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5196272eaf0846a186088131daa36cc9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:12.768128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5196272eaf0846a186088131daa36cc9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:12.797944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 750/1479] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:12 2022] DELETE /identity/v3/users/5196272eaf0846a186088131daa36cc9 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:12.802810 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ca111-c4fa-4648-9531-8ed51a4337c4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:12.803382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ca111-c4fa-4648-9531-8ed51a4337c4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:12.803382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ca111-c4fa-4648-9531-8ed51a4337c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:12.825107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-849ca111-c4fa-4648-9531-8ed51a4337c4 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:12.826568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-849ca111-c4fa-4648-9531-8ed51a4337c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:12.916989 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 730/1480] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:12.927502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:12.930215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:13.005069 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d96e3-ac17-4b87-a6f7-d975d9d1a740 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:13.011817 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa481c-e54c-4397-8c6c-ab8fddbb3ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:13.012324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa481c-e54c-4397-8c6c-ab8fddbb3ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:13.012607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa481c-e54c-4397-8c6c-ab8fddbb3ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:13.012880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa481c-e54c-4397-8c6c-ab8fddbb3ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:13.024104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa481c-e54c-4397-8c6c-ab8fddbb3ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:13.025832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa481c-e54c-4397-8c6c-ab8fddbb3ba9 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:13.037020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 751/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:13.386938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ca111-c4fa-4648-9531-8ed51a4337c4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:13.388724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-849ca111-c4fa-4648-9531-8ed51a4337c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:13.414217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ca111-c4fa-4648-9531-8ed51a4337c4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:13.419461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:13.421727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:13.422578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:13.422998 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] PATH_INFO: `/v3/users/a441584aa5a6409e86b1f84f8bcb28a4` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:13.424291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a441584aa5a6409e86b1f84f8bcb28a4)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:13.428230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:13.489092 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa481c-e54c-4397-8c6c-ab8fddbb3ba9 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:13.491283 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bdaa481c-e54c-4397-8c6c-ab8fddbb3ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:13.509459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa481c-e54c-4397-8c6c-ab8fddbb3ba9 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:13.519250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:13.519250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:13.519250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:13.519250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] PATH_INFO: `/v3/projects/c866b8e05ef84edbae2b1a0baf04f3a3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:13.519837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c866b8e05ef84edbae2b1a0baf04f3a3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:13.520726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:13.685473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:13.694055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c866b8e05ef84edbae2b1a0baf04f3a3', 'request_id': 'req-8a6bdede-1c87-4744-a8af-604bb3977b41'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.697769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a441584aa5a6409e86b1f84f8bcb28a4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.698642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a441584aa5a6409e86b1f84f8bcb28a4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.708127 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a441584aa5a6409e86b1f84f8bcb28a4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.708854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a441584aa5a6409e86b1f84f8bcb28a4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.762627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c866b8e05ef84edbae2b1a0baf04f3a3', 'request_id': 'req-8a6bdede-1c87-4744-a8af-604bb3977b41'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.770769 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c866b8e05ef84edbae2b1a0baf04f3a3', 'request_id': 'req-8a6bdede-1c87-4744-a8af-604bb3977b41'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.782230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] The token cache is being invalidate because project c866b8e05ef84edbae2b1a0baf04f3a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:13.782702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.783554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.785413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 752/1482] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:13 2022] DELETE /identity/v3/projects/c866b8e05ef84edbae2b1a0baf04f3a3 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:13.789083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:13.790758 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:13.830945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c6892b3-b128-4845-99a0-36fc9354e471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a441584aa5a6409e86b1f84f8bcb28a4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:13.843678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:13.856890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 731/1483] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:13 2022] DELETE /identity/v3/users/a441584aa5a6409e86b1f84f8bcb28a4 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:13.861975 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8766c05-bc29-4ac0-98f2-43e785ba1c6b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:13.862315 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8766c05-bc29-4ac0-98f2-43e785ba1c6b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:13.862713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8766c05-bc29-4ac0-98f2-43e785ba1c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:13.863792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6bdede-1c87-4744-a8af-604bb3977b41 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:13.882271 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a8766c05-bc29-4ac0-98f2-43e785ba1c6b None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:13.883506 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a8766c05-bc29-4ac0-98f2-43e785ba1c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:13.921991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:13.922645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:13.923448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:13.923726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] PATH_INFO: `/v3/users/54448e6c6de44e87b0994365067664ab` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:13.924898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] RBAC: Authorizing `identity:delete_user(user_id=54448e6c6de44e87b0994365067664ab)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:13.927182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:13.946186 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 732/1484] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:13.956623 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8766c05-bc29-4ac0-98f2-43e785ba1c6b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:13.958029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a8766c05-bc29-4ac0-98f2-43e785ba1c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:14.013430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8766c05-bc29-4ac0-98f2-43e785ba1c6b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:14.017164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111207d3-a1c2-4259-9941-bee095d4f63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:14.017890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111207d3-a1c2-4259-9941-bee095d4f63a service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:14.018171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111207d3-a1c2-4259-9941-bee095d4f63a service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:14.018443 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111207d3-a1c2-4259-9941-bee095d4f63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:14.028434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111207d3-a1c2-4259-9941-bee095d4f63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:14.029913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111207d3-a1c2-4259-9941-bee095d4f63a service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:14.039456 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 733/1485] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:14.111085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54448e6c6de44e87b0994365067664ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.123289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54448e6c6de44e87b0994365067664ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.124051 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54448e6c6de44e87b0994365067664ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.125887 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54448e6c6de44e87b0994365067664ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.184224 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54448e6c6de44e87b0994365067664ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.198562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:14.209924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 753/1486] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:13 2022] DELETE /identity/v3/users/54448e6c6de44e87b0994365067664ab => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:14.520043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111207d3-a1c2-4259-9941-bee095d4f63a service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:14.521782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-111207d3-a1c2-4259-9941-bee095d4f63a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:14.540017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111207d3-a1c2-4259-9941-bee095d4f63a service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:14.612229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:14.613016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:14.613896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:14.614182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] PATH_INFO: `/v3/projects/7b5abd4b70eb4b18af00352a0f33c924` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:14.615343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5abd4b70eb4b18af00352a0f33c924)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:14.617371 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:14.731009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=4,keystone:SELECT=96 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:14.738738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5abd4b70eb4b18af00352a0f33c924', 'request_id': 'req-c01f2cdd-eff8-42f5-9c23-89440e60ad23'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.745303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5abd4b70eb4b18af00352a0f33c924', 'request_id': 'req-c01f2cdd-eff8-42f5-9c23-89440e60ad23'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.861114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5abd4b70eb4b18af00352a0f33c924', 'request_id': 'req-c01f2cdd-eff8-42f5-9c23-89440e60ad23'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.876066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:14.877710 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:14.903030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a6a5c-0ed8-4edb-b538-a1eda911a7db admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:14.908991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] The token cache is being invalidate because project 7b5abd4b70eb4b18af00352a0f33c924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:14.909340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.911077 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:14.920184 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 734/1487] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:14 2022] DELETE /identity/v3/projects/7b5abd4b70eb4b18af00352a0f33c924 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:14.921895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:14.926682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:14.982323 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:14.988369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:14.988369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:14.988369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] PATH_INFO: `/v3/users/c553103d2ec44e37a361aad3559ce6df` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:14.988369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c553103d2ec44e37a361aad3559ce6df)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:14.988369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:15.006094 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f2cdd-eff8-42f5-9c23-89440e60ad23 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:15.022737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:15.023412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.023695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.023972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] PATH_INFO: `/v3/projects/44183268008e4dd39615e350aa86c56d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.025113 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44183268008e4dd39615e350aa86c56d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:15.027082 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:15.151999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44183268008e4dd39615e350aa86c56d', 'request_id': 'req-4198a5fc-b874-46ec-8185-ad7f1ba99a65'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.152218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c553103d2ec44e37a361aad3559ce6df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.159070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44183268008e4dd39615e350aa86c56d', 'request_id': 'req-4198a5fc-b874-46ec-8185-ad7f1ba99a65'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.160132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c553103d2ec44e37a361aad3559ce6df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.162239 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c553103d2ec44e37a361aad3559ce6df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.162239 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c553103d2ec44e37a361aad3559ce6df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.207136 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44183268008e4dd39615e350aa86c56d', 'request_id': 'req-4198a5fc-b874-46ec-8185-ad7f1ba99a65'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.218451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] The token cache is being invalidate because project 44183268008e4dd39615e350aa86c56d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:15.218743 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.219387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.221119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 735/1488] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:14 2022] DELETE /identity/v3/projects/44183268008e4dd39615e350aa86c56d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:15.224672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:15.224672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:15.241445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54ab1d97-460d-4337-a1b1-61a9f1fd41cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c553103d2ec44e37a361aad3559ce6df'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.265231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 754/1489] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:14 2022] DELETE /identity/v3/users/c553103d2ec44e37a361aad3559ce6df => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:15.266096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9fd7c-900b-430d-ad0c-e5ceb7470b96 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.266518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9fd7c-900b-430d-ad0c-e5ceb7470b96 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.266880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9fd7c-900b-430d-ad0c-e5ceb7470b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.287624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6cc9fd7c-900b-430d-ad0c-e5ceb7470b96 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:15.288859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6cc9fd7c-900b-430d-ad0c-e5ceb7470b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:15.299456 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:15.303665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198a5fc-b874-46ec-8185-ad7f1ba99a65 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:15.361302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 755/1490] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:15.363004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82c011-f534-402a-8bd8-e2fdd8531f36 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.363463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82c011-f534-402a-8bd8-e2fdd8531f36 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.363954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82c011-f534-402a-8bd8-e2fdd8531f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.377832 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:15.378516 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.378795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.379069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] PATH_INFO: `/v3/users/74169ab9e3c04f90ad0594c2c4e0a7bb` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.380232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74169ab9e3c04f90ad0594c2c4e0a7bb)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:15.382191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:15.387968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bb82c011-f534-402a-8bd8-e2fdd8531f36 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:15.390277 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bb82c011-f534-402a-8bd8-e2fdd8531f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:15.405550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 756/1491] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:15.407172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec797ae-8113-4933-9916-b51d622c12b3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.407581 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec797ae-8113-4933-9916-b51d622c12b3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.408079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec797ae-8113-4933-9916-b51d622c12b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.424283 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:15.430627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0ec797ae-8113-4933-9916-b51d622c12b3 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:15.431883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0ec797ae-8113-4933-9916-b51d622c12b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:15.440512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 757/1492] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:15.444626 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec797ae-8113-4933-9916-b51d622c12b3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:15.445971 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0ec797ae-8113-4933-9916-b51d622c12b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:15.505365 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec797ae-8113-4933-9916-b51d622c12b3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:15.509588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1af711-7ec4-455c-99d0-430f86eab5ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:15.514781 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1af711-7ec4-455c-99d0-430f86eab5ff service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.514781 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1af711-7ec4-455c-99d0-430f86eab5ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.514781 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1af711-7ec4-455c-99d0-430f86eab5ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.520014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1af711-7ec4-455c-99d0-430f86eab5ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:15.521557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1af711-7ec4-455c-99d0-430f86eab5ff service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:15.530955 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 758/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:15.531856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1af711-7ec4-455c-99d0-430f86eab5ff service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:15.543172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1af711-7ec4-455c-99d0-430f86eab5ff service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:15.551253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:15.552669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.552944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.553246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.553969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:15.554278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:15.556288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:15.558688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74169ab9e3c04f90ad0594c2c4e0a7bb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.559512 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74169ab9e3c04f90ad0594c2c4e0a7bb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.575755 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 759/1494] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:15.581744 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:15.584183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:15.584311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74169ab9e3c04f90ad0594c2c4e0a7bb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.585106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74169ab9e3c04f90ad0594c2c4e0a7bb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.599742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cfbc3-6a4b-4038-b55a-7df216a655af admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:15.604387 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ba0dd-5e0b-48f8-a4ff-7a8b519bf623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:15.605014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ba0dd-5e0b-48f8-a4ff-7a8b519bf623 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.605320 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ba0dd-5e0b-48f8-a4ff-7a8b519bf623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.605591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ba0dd-5e0b-48f8-a4ff-7a8b519bf623 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.606242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ba0dd-5e0b-48f8-a4ff-7a8b519bf623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:15.608153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ba0dd-5e0b-48f8-a4ff-7a8b519bf623 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:15.664666 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74169ab9e3c04f90ad0594c2c4e0a7bb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:15.684026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 736/1495] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:15 2022] DELETE /identity/v3/users/74169ab9e3c04f90ad0594c2c4e0a7bb => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:15.694883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 760/1496] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:15 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:15.698981 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:15.764368 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c131c1e-561c-4efb-b9e1-b51e7b02b5f1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:15.802592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:15.837173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aad587-3b2d-45c3-b051-22399622c467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:15.837492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aad587-3b2d-45c3-b051-22399622c467 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.837916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aad587-3b2d-45c3-b051-22399622c467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.838243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aad587-3b2d-45c3-b051-22399622c467 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.839068 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aad587-3b2d-45c3-b051-22399622c467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:15.841234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aad587-3b2d-45c3-b051-22399622c467 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:15.920952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 737/1497] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:15 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:15.924782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ba0dd-5e0b-48f8-a4ff-7a8b519bf623 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:15.936017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ba0dd-5e0b-48f8-a4ff-7a8b519bf623 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:15.941983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5637a-1863-40f4-9056-fb789fd906a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:15.942736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5637a-1863-40f4-9056-fb789fd906a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.943020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5637a-1863-40f4-9056-fb789fd906a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.945618 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5637a-1863-40f4-9056-fb789fd906a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.945618 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5637a-1863-40f4-9056-fb789fd906a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:15.945877 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5637a-1863-40f4-9056-fb789fd906a9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:15.960646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 761/1498] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:15.962069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aad587-3b2d-45c3-b051-22399622c467 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:15.989784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aad587-3b2d-45c3-b051-22399622c467 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:15.997291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456fe7c-63b5-4ab7-85bb-f2be99f9b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:15.998044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456fe7c-63b5-4ab7-85bb-f2be99f9b547 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:15.998333 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456fe7c-63b5-4ab7-85bb-f2be99f9b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:15.998618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456fe7c-63b5-4ab7-85bb-f2be99f9b547 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:15.999273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456fe7c-63b5-4ab7-85bb-f2be99f9b547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:16.000884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456fe7c-63b5-4ab7-85bb-f2be99f9b547 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:16.012488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 738/1499] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:16.016324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5637a-1863-40f4-9056-fb789fd906a9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:16.039903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5637a-1863-40f4-9056-fb789fd906a9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:16.046549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf626ca-4215-46af-94e7-51e128b0d8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:16.047250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf626ca-4215-46af-94e7-51e128b0d8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:16.047534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf626ca-4215-46af-94e7-51e128b0d8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:16.048352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf626ca-4215-46af-94e7-51e128b0d8d4 admin admin] PATH_INFO: `/v3/projects/ba04db30b0c045fc9c4f581067275a2e/users/6222220c13e841af8e02056ecd7b668e/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:16.065800 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf626ca-4215-46af-94e7-51e128b0d8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba04db30b0c045fc9c4f581067275a2e, user_id=6222220c13e841af8e02056ecd7b668e, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:16.068498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf626ca-4215-46af-94e7-51e128b0d8d4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:16.090696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456fe7c-63b5-4ab7-85bb-f2be99f9b547 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:16.092520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c456fe7c-63b5-4ab7-85bb-f2be99f9b547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:16.106507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456fe7c-63b5-4ab7-85bb-f2be99f9b547 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:16.112821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:16.113498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:16.113778 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:16.114056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] PATH_INFO: `/v3/projects/30ffd49709c54f678e177548ffba085d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:16.115729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ffd49709c54f678e177548ffba085d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:16.117680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:16.175040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 762/1500] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:16 2022] PUT /identity/v3/projects/ba04db30b0c045fc9c4f581067275a2e/users/6222220c13e841af8e02056ecd7b668e/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:16.176622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf626ca-4215-46af-94e7-51e128b0d8d4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:16.180025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cbf626ca-4215-46af-94e7-51e128b0d8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:16.266071 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ffd49709c54f678e177548ffba085d', 'request_id': 'req-14bfaf82-0cf7-4377-b724-206bb32271c0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.277286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ffd49709c54f678e177548ffba085d', 'request_id': 'req-14bfaf82-0cf7-4377-b724-206bb32271c0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.278947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf626ca-4215-46af-94e7-51e128b0d8d4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:16.318616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:16.336737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:16.337413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:16.337696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:16.337975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] PATH_INFO: `/v3/projects/aa6224ee09ae4e868c23a0da09738468` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:16.339104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa6224ee09ae4e868c23a0da09738468)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:16.341041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:16.343903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ffd49709c54f678e177548ffba085d', 'request_id': 'req-14bfaf82-0cf7-4377-b724-206bb32271c0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.358822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] The token cache is being invalidate because project 30ffd49709c54f678e177548ffba085d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:16.359118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.359828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bfaf82-0cf7-4377-b724-206bb32271c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.362682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 739/1501] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:16 2022] DELETE /identity/v3/projects/30ffd49709c54f678e177548ffba085d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:16.368246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467654a-5928-4a49-b213-e35c2a731fd9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:16.368608 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467654a-5928-4a49-b213-e35c2a731fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:16.368831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467654a-5928-4a49-b213-e35c2a731fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:16.397526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2467654a-5928-4a49-b213-e35c2a731fd9 None None] MFA Rules not processed for user `6222220c13e841af8e02056ecd7b668e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:16.398869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2467654a-5928-4a49-b213-e35c2a731fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:16.465925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 740/1502] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:16.466781 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467654a-5928-4a49-b213-e35c2a731fd9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:16.468076 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2467654a-5928-4a49-b213-e35c2a731fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:16.470533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa6224ee09ae4e868c23a0da09738468', 'request_id': 'req-100055f2-d214-4ca7-a89f-ec4edad46154'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.482258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467654a-5928-4a49-b213-e35c2a731fd9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:16.501863 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:16.544180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:16.544917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:16.545223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:16.545525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] PATH_INFO: `/v3/users/f3e7971a94804ae289e3f33fb6a93e26` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:16.548064 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e7971a94804ae289e3f33fb6a93e26)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:16.549158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:16.567790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa6224ee09ae4e868c23a0da09738468', 'request_id': 'req-100055f2-d214-4ca7-a89f-ec4edad46154'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.575644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa6224ee09ae4e868c23a0da09738468', 'request_id': 'req-100055f2-d214-4ca7-a89f-ec4edad46154'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.590346 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] The token cache is being invalidate because project aa6224ee09ae4e868c23a0da09738468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:16.590688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.591326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.593242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 763/1503] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:16 2022] DELETE /identity/v3/projects/aa6224ee09ae4e868c23a0da09738468 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:16.593931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:16.595408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:16.617531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100055f2-d214-4ca7-a89f-ec4edad46154 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:16.622226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1b74a-d317-48b2-98a8-a51bce00657a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:16.622894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1b74a-d317-48b2-98a8-a51bce00657a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:16.623188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1b74a-d317-48b2-98a8-a51bce00657a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:16.623465 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1b74a-d317-48b2-98a8-a51bce00657a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:16.624101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1b74a-d317-48b2-98a8-a51bce00657a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:16.626625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1b74a-d317-48b2-98a8-a51bce00657a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:16.752718 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 764/1504] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:16 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:16.752718 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1b74a-d317-48b2-98a8-a51bce00657a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:16.757214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-72f1b74a-d317-48b2-98a8-a51bce00657a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:16.787897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e7971a94804ae289e3f33fb6a93e26'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.788600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e7971a94804ae289e3f33fb6a93e26'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.796689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e7971a94804ae289e3f33fb6a93e26'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.797377 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e7971a94804ae289e3f33fb6a93e26'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.821989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1b74a-d317-48b2-98a8-a51bce00657a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:16.826628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:16.827259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:16.827550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:16.827832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] PATH_INFO: `/v3/users/c9950eb4b476471e8dae0aae10a1cbac` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:16.828990 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9950eb4b476471e8dae0aae10a1cbac)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:16.831558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:16.864068 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e7971a94804ae289e3f33fb6a93e26'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:16.881923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 741/1505] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:16 2022] DELETE /identity/v3/users/f3e7971a94804ae289e3f33fb6a93e26 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:16.882703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:16.884137 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:16.952838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59860b-4c54-42d2-acfd-00aa72647a04 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:16.958337 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078a86b-842c-4820-9144-585579e73dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:16.958981 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a86b-842c-4820-9144-585579e73dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:16.959258 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a86b-842c-4820-9144-585579e73dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:16.959529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a86b-842c-4820-9144-585579e73dbd admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:16.960166 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078a86b-842c-4820-9144-585579e73dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:16.962122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078a86b-842c-4820-9144-585579e73dbd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:17.000873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9950eb4b476471e8dae0aae10a1cbac'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:17.003639 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:17.015511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9950eb4b476471e8dae0aae10a1cbac'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:17.016351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9950eb4b476471e8dae0aae10a1cbac'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:17.018691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9950eb4b476471e8dae0aae10a1cbac'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:17.042626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 742/1506] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:16 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:17.046423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078a86b-842c-4820-9144-585579e73dbd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:17.051925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d078a86b-842c-4820-9144-585579e73dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:17.061892 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078a86b-842c-4820-9144-585579e73dbd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:17.066930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae451019-bce7-4006-b07a-cefb6ac899e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:17.067547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451019-bce7-4006-b07a-cefb6ac899e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:17.067827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451019-bce7-4006-b07a-cefb6ac899e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:17.068137 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451019-bce7-4006-b07a-cefb6ac899e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:17.068830 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae451019-bce7-4006-b07a-cefb6ac899e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:17.086684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae451019-bce7-4006-b07a-cefb6ac899e8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:17.086684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 743/1507] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:17.087079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae451019-bce7-4006-b07a-cefb6ac899e8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:17.097888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae451019-bce7-4006-b07a-cefb6ac899e8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:17.102728 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fef19-4100-494a-81ce-51156260cb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:17.109605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fef19-4100-494a-81ce-51156260cb9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:17.109605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fef19-4100-494a-81ce-51156260cb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:17.109605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fef19-4100-494a-81ce-51156260cb9d admin admin] PATH_INFO: `/v3/projects/81ca09a0e852463f9a312f5043563dde/users/5fd2beb18ecc4202a5cdea346af2fd0c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:17.123399 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fef19-4100-494a-81ce-51156260cb9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ca09a0e852463f9a312f5043563dde, user_id=5fd2beb18ecc4202a5cdea346af2fd0c, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:17.127007 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fef19-4100-494a-81ce-51156260cb9d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:17.132301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9950eb4b476471e8dae0aae10a1cbac'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:17.147891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 765/1508] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:16 2022] DELETE /identity/v3/users/c9950eb4b476471e8dae0aae10a1cbac => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:17.207974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:17.220350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 744/1509] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:17 2022] PUT /identity/v3/projects/81ca09a0e852463f9a312f5043563dde/users/5fd2beb18ecc4202a5cdea346af2fd0c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:17.222834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:17.300614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e014c28-0ad8-4f45-b1bf-1a86edec346a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:17.381841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3695c-c303-4887-af54-3491b25b5794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:17.382785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3695c-c303-4887-af54-3491b25b5794 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:17.382942 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3695c-c303-4887-af54-3491b25b5794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:17.383351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3695c-c303-4887-af54-3491b25b5794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:17.384073 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3695c-c303-4887-af54-3491b25b5794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:17.385948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3695c-c303-4887-af54-3491b25b5794 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:17.398285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 766/1510] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:17.402855 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fef19-4100-494a-81ce-51156260cb9d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:17.417218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fef19-4100-494a-81ce-51156260cb9d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:17.424720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd981279-6ac8-4a53-8937-30641e263782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:17.425857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd981279-6ac8-4a53-8937-30641e263782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:17.426320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd981279-6ac8-4a53-8937-30641e263782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:17.427533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd981279-6ac8-4a53-8937-30641e263782 admin admin] PATH_INFO: `/v3/projects/81ca09a0e852463f9a312f5043563dde/users/5fd2beb18ecc4202a5cdea346af2fd0c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:17.429837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd981279-6ac8-4a53-8937-30641e263782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ca09a0e852463f9a312f5043563dde, user_id=5fd2beb18ecc4202a5cdea346af2fd0c, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:17.432755 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd981279-6ac8-4a53-8937-30641e263782 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:17.448919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 745/1511] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:17 2022] PUT /identity/v3/projects/81ca09a0e852463f9a312f5043563dde/users/5fd2beb18ecc4202a5cdea346af2fd0c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:17.460383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3efdc-6cde-40bb-9de8-b5a6bce11a8e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:17.460944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3efdc-6cde-40bb-9de8-b5a6bce11a8e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:17.461248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3efdc-6cde-40bb-9de8-b5a6bce11a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:17.497454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fcf3efdc-6cde-40bb-9de8-b5a6bce11a8e None None] MFA Rules not processed for user `5fd2beb18ecc4202a5cdea346af2fd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:17.498841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fcf3efdc-6cde-40bb-9de8-b5a6bce11a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:17.637959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 767/1512] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:17.650759 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd981279-6ac8-4a53-8937-30641e263782 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:17.653632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bd981279-6ac8-4a53-8937-30641e263782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:17.787055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd981279-6ac8-4a53-8937-30641e263782 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:17.791613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247963f-9f83-4b2c-bb64-6149bffeb5a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:17.792243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247963f-9f83-4b2c-bb64-6149bffeb5a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:17.792528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247963f-9f83-4b2c-bb64-6149bffeb5a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:17.792801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247963f-9f83-4b2c-bb64-6149bffeb5a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:17.807485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247963f-9f83-4b2c-bb64-6149bffeb5a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:17.809100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247963f-9f83-4b2c-bb64-6149bffeb5a1 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:17.821908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 746/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:17.866263 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:17.889316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3efdc-6cde-40bb-9de8-b5a6bce11a8e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:17.891394 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fcf3efdc-6cde-40bb-9de8-b5a6bce11a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:17.971015 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247963f-9f83-4b2c-bb64-6149bffeb5a1 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:18.036730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3efdc-6cde-40bb-9de8-b5a6bce11a8e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:18.045021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:18.047631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.049147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.051235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] PATH_INFO: `/v3/projects/b58ec448cefd4a20ad29e4864c011449` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.057521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58ec448cefd4a20ad29e4864c011449)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:18.064145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:18.106927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247963f-9f83-4b2c-bb64-6149bffeb5a1 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:18.111796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:18.112436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.112719 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.112995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] PATH_INFO: `/v3/users/6222220c13e841af8e02056ecd7b668e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.114209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6222220c13e841af8e02056ecd7b668e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:18.116143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:18.213840 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58ec448cefd4a20ad29e4864c011449', 'request_id': 'req-288e6a96-db86-4f04-9498-dfef3ac5e241'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.304672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58ec448cefd4a20ad29e4864c011449', 'request_id': 'req-288e6a96-db86-4f04-9498-dfef3ac5e241'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.311815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58ec448cefd4a20ad29e4864c011449', 'request_id': 'req-288e6a96-db86-4f04-9498-dfef3ac5e241'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.321082 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6222220c13e841af8e02056ecd7b668e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.321750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6222220c13e841af8e02056ecd7b668e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.324863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] The token cache is being invalidate because project b58ec448cefd4a20ad29e4864c011449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:18.325162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.325829 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.327335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 768/1514] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:17 2022] DELETE /identity/v3/projects/b58ec448cefd4a20ad29e4864c011449 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:18.328139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:18.329564 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:18.333021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6222220c13e841af8e02056ecd7b668e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.333718 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6222220c13e841af8e02056ecd7b668e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.395220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4f099549-ea2a-405d-839f-750e8077136f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6222220c13e841af8e02056ecd7b668e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.395481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e6a96-db86-4f04-9498-dfef3ac5e241 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:18.400368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:18.401100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.401421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.401726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] PATH_INFO: `/v3/projects/1ae15a69bb48499097c9b828452c1e0f` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.406178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae15a69bb48499097c9b828452c1e0f)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:18.406178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:18.414395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 747/1515] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:17 2022] DELETE /identity/v3/users/6222220c13e841af8e02056ecd7b668e => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:18.419260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93c084-14c5-40aa-98b2-c6db21678f67 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.419534 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93c084-14c5-40aa-98b2-c6db21678f67 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.419811 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93c084-14c5-40aa-98b2-c6db21678f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.429179 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:18.441165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6d93c084-14c5-40aa-98b2-c6db21678f67 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:18.442422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6d93c084-14c5-40aa-98b2-c6db21678f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:18.500121 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 748/1516] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:18.502126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f66106-5f30-4593-9d4e-e582279850c2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.502410 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f66106-5f30-4593-9d4e-e582279850c2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.502955 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f66106-5f30-4593-9d4e-e582279850c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.512274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:18.522064 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-43f66106-5f30-4593-9d4e-e582279850c2 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:18.523278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-43f66106-5f30-4593-9d4e-e582279850c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:18.528394 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae15a69bb48499097c9b828452c1e0f', 'request_id': 'req-ba64caa0-22e9-44fd-be5f-e9ef912923b4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.589290 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 749/1517] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:18.590522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6dccb-afdc-4e17-a0df-0b84035fbc55 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.590815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6dccb-afdc-4e17-a0df-0b84035fbc55 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.591099 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6dccb-afdc-4e17-a0df-0b84035fbc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.596351 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae15a69bb48499097c9b828452c1e0f', 'request_id': 'req-ba64caa0-22e9-44fd-be5f-e9ef912923b4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.604647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae15a69bb48499097c9b828452c1e0f', 'request_id': 'req-ba64caa0-22e9-44fd-be5f-e9ef912923b4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.613586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-60f6dccb-afdc-4e17-a0df-0b84035fbc55 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:18.614819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-60f6dccb-afdc-4e17-a0df-0b84035fbc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:18.618614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] The token cache is being invalidate because project 1ae15a69bb48499097c9b828452c1e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:18.618923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.619662 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:18.625920 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 769/1518] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:18 2022] DELETE /identity/v3/projects/1ae15a69bb48499097c9b828452c1e0f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:18.625920 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:18.625920 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:18.629167 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 750/1519] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:18.629325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7493cd9-ffd3-40df-8cc8-7876ca176841 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.629533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7493cd9-ffd3-40df-8cc8-7876ca176841 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.629872 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7493cd9-ffd3-40df-8cc8-7876ca176841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.657984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a7493cd9-ffd3-40df-8cc8-7876ca176841 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:18.659382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a7493cd9-ffd3-40df-8cc8-7876ca176841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:18.668768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 751/1520] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:18.676723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7493cd9-ffd3-40df-8cc8-7876ca176841 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:18.676723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a7493cd9-ffd3-40df-8cc8-7876ca176841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:18.686133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7493cd9-ffd3-40df-8cc8-7876ca176841 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:18.694161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63d221-845c-40e5-a3fa-c67517a14a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:18.694927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63d221-845c-40e5-a3fa-c67517a14a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.695207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63d221-845c-40e5-a3fa-c67517a14a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.695482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63d221-845c-40e5-a3fa-c67517a14a03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.696172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63d221-845c-40e5-a3fa-c67517a14a03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:18.696476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63d221-845c-40e5-a3fa-c67517a14a03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:18.698433 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63d221-845c-40e5-a3fa-c67517a14a03 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:18.700699 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64caa0-22e9-44fd-be5f-e9ef912923b4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:18.705014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a2783-d270-448f-b6a8-a922f07d124b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:18.705834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a2783-d270-448f-b6a8-a922f07d124b service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.707056 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a2783-d270-448f-b6a8-a922f07d124b service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.707502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a2783-d270-448f-b6a8-a922f07d124b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.709782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-605a2783-d270-448f-b6a8-a922f07d124b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:18.714721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 752/1521] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:18.714721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2227889-db50-449d-851b-7ba3b38444fb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.714721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2227889-db50-449d-851b-7ba3b38444fb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.714721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2227889-db50-449d-851b-7ba3b38444fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.729022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a2783-d270-448f-b6a8-a922f07d124b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:18.730702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a2783-d270-448f-b6a8-a922f07d124b service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:18.736660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e2227889-db50-449d-851b-7ba3b38444fb None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:18.737978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e2227889-db50-449d-851b-7ba3b38444fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:18.741273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 770/1522] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:18.742119 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a2783-d270-448f-b6a8-a922f07d124b service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:18.746160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 753/1523] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:18.750294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2227889-db50-449d-851b-7ba3b38444fb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:18.752149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a2783-d270-448f-b6a8-a922f07d124b service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:18.757544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ab6d1-1372-4bfb-824c-e3a598f9464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:18.758248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ab6d1-1372-4bfb-824c-e3a598f9464a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.758647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ab6d1-1372-4bfb-824c-e3a598f9464a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.759021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ab6d1-1372-4bfb-824c-e3a598f9464a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.759753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ab6d1-1372-4bfb-824c-e3a598f9464a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:18.760246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2227889-db50-449d-851b-7ba3b38444fb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:18.761907 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ab6d1-1372-4bfb-824c-e3a598f9464a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:18.765108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5f0db-4555-4946-886e-34d844278d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:18.765824 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5f0db-4555-4946-886e-34d844278d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.766311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5f0db-4555-4946-886e-34d844278d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.766544 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5f0db-4555-4946-886e-34d844278d82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.767319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5f0db-4555-4946-886e-34d844278d82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:18.767703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5f0db-4555-4946-886e-34d844278d82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:18.769404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5f0db-4555-4946-886e-34d844278d82 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:18.797603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 754/1524] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:04:18.802660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5f0db-4555-4946-886e-34d844278d82 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:18.813431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5f0db-4555-4946-886e-34d844278d82 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:18.819040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1857de-0586-4229-8338-bdf0a01b4b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:18.819755 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1857de-0586-4229-8338-bdf0a01b4b30 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:18.820276 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1857de-0586-4229-8338-bdf0a01b4b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:18.820636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1857de-0586-4229-8338-bdf0a01b4b30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:18.821401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1857de-0586-4229-8338-bdf0a01b4b30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:18.823514 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1857de-0586-4229-8338-bdf0a01b4b30 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:18.858993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 771/1525] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:18 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:18.864431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ab6d1-1372-4bfb-824c-e3a598f9464a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:18.917701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 755/1526] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:18 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:18.918935 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1857de-0586-4229-8338-bdf0a01b4b30 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:18.974220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:18.976386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ab6d1-1372-4bfb-824c-e3a598f9464a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.026420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1857de-0586-4229-8338-bdf0a01b4b30 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.035460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640ac4d-4604-4cfb-a077-0588b44d1f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.036199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640ac4d-4604-4cfb-a077-0588b44d1f88 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.036584 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640ac4d-4604-4cfb-a077-0588b44d1f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.036965 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640ac4d-4604-4cfb-a077-0588b44d1f88 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.037874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640ac4d-4604-4cfb-a077-0588b44d1f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.040208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640ac4d-4604-4cfb-a077-0588b44d1f88 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.057089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:19.073332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c94cc-f73b-484e-8f25-d88e941ecaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.074181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c94cc-f73b-484e-8f25-d88e941ecaf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.074570 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c94cc-f73b-484e-8f25-d88e941ecaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.074982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c94cc-f73b-484e-8f25-d88e941ecaf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.076925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c94cc-f73b-484e-8f25-d88e941ecaf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.077730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c94cc-f73b-484e-8f25-d88e941ecaf8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.140583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 756/1527] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:18 2022] POST /identity/v3/users => generated 292 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:19.144735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640ac4d-4604-4cfb-a077-0588b44d1f88 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.162864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640ac4d-4604-4cfb-a077-0588b44d1f88 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.166271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 772/1528] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:18 2022] POST /identity/v3/users => generated 294 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:19.167690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c94cc-f73b-484e-8f25-d88e941ecaf8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.168527 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87071c4a-8b1b-4c07-abc6-741f2c2690f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.169158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87071c4a-8b1b-4c07-abc6-741f2c2690f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.169491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87071c4a-8b1b-4c07-abc6-741f2c2690f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.169769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87071c4a-8b1b-4c07-abc6-741f2c2690f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.170427 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87071c4a-8b1b-4c07-abc6-741f2c2690f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.172256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87071c4a-8b1b-4c07-abc6-741f2c2690f1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.180057 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c94cc-f73b-484e-8f25-d88e941ecaf8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.185342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 757/1529] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:19.189325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160baf7-3bc4-4a49-97a9-f83a5734ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.190108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160baf7-3bc4-4a49-97a9-f83a5734ffb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.190509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160baf7-3bc4-4a49-97a9-f83a5734ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.190884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160baf7-3bc4-4a49-97a9-f83a5734ffb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.190993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87071c4a-8b1b-4c07-abc6-741f2c2690f1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.191951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160baf7-3bc4-4a49-97a9-f83a5734ffb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.193750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160baf7-3bc4-4a49-97a9-f83a5734ffb6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.203433 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87071c4a-8b1b-4c07-abc6-741f2c2690f1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.209428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 773/1530] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:19.210853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160baf7-3bc4-4a49-97a9-f83a5734ffb6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.213724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ba430-157f-4569-82f6-94e07af759a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.214358 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ba430-157f-4569-82f6-94e07af759a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.214644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ba430-157f-4569-82f6-94e07af759a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.214927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ba430-157f-4569-82f6-94e07af759a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.215581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ba430-157f-4569-82f6-94e07af759a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.223001 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160baf7-3bc4-4a49-97a9-f83a5734ffb6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.223535 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ba430-157f-4569-82f6-94e07af759a5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.238372 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 758/1531] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:19.242778 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ba430-157f-4569-82f6-94e07af759a5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.252868 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab319a2-a0d5-4761-ba31-fc88929faa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.254765 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab319a2-a0d5-4761-ba31-fc88929faa29 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.256307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab319a2-a0d5-4761-ba31-fc88929faa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.256698 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab319a2-a0d5-4761-ba31-fc88929faa29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.258445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab319a2-a0d5-4761-ba31-fc88929faa29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.260229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab319a2-a0d5-4761-ba31-fc88929faa29 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.260347 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ba430-157f-4569-82f6-94e07af759a5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.267571 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c77aad-711f-4642-a592-cc6b0fba3fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.268498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c77aad-711f-4642-a592-cc6b0fba3fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.268612 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c77aad-711f-4642-a592-cc6b0fba3fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.269032 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c77aad-711f-4642-a592-cc6b0fba3fd0 admin admin] PATH_INFO: `/v3/projects/77c355899b8540d99ba3638c329d8def/users/66fd41053c8845a3bb10e8899af2defb/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.283684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 774/1532] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:19.283684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab319a2-a0d5-4761-ba31-fc88929faa29 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.286319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-eab319a2-a0d5-4761-ba31-fc88929faa29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:19.290558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c77aad-711f-4642-a592-cc6b0fba3fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77c355899b8540d99ba3638c329d8def, user_id=66fd41053c8845a3bb10e8899af2defb, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.293707 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c77aad-711f-4642-a592-cc6b0fba3fd0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.301416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab319a2-a0d5-4761-ba31-fc88929faa29 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.308438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.308438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.308438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.308709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] PATH_INFO: `/v3/projects/ba04db30b0c045fc9c4f581067275a2e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.311297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba04db30b0c045fc9c4f581067275a2e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.313910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.386065 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 759/1533] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:19 2022] PUT /identity/v3/projects/77c355899b8540d99ba3638c329d8def/users/66fd41053c8845a3bb10e8899af2defb/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:19.386910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c77aad-711f-4642-a592-cc6b0fba3fd0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.452989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba04db30b0c045fc9c4f581067275a2e', 'request_id': 'req-aade689f-053a-4691-8685-0c4342414b48'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:19.463481 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c77aad-711f-4642-a592-cc6b0fba3fd0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.523177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2c19a-8f29-4aa9-b5ae-b760f4270964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.523944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2c19a-8f29-4aa9-b5ae-b760f4270964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.524227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2c19a-8f29-4aa9-b5ae-b760f4270964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.524540 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2c19a-8f29-4aa9-b5ae-b760f4270964 admin admin] PATH_INFO: `/v3/projects/7b1d101acb9549efa30ab0b0a25c32a4/users/3466eb5c5f4f465eb5b02fd297ec7298/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.540321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2c19a-8f29-4aa9-b5ae-b760f4270964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1d101acb9549efa30ab0b0a25c32a4, user_id=3466eb5c5f4f465eb5b02fd297ec7298, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.543692 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2c19a-8f29-4aa9-b5ae-b760f4270964 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.545112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba04db30b0c045fc9c4f581067275a2e', 'request_id': 'req-aade689f-053a-4691-8685-0c4342414b48'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:19.553907 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba04db30b0c045fc9c4f581067275a2e', 'request_id': 'req-aade689f-053a-4691-8685-0c4342414b48'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:19.566025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] The token cache is being invalidate because project ba04db30b0c045fc9c4f581067275a2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:19.566304 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:19.567012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aade689f-053a-4691-8685-0c4342414b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:19.568734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 775/1534] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:19 2022] DELETE /identity/v3/projects/ba04db30b0c045fc9c4f581067275a2e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:19.569938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ad4d-7c41-425f-89f8-ed0f2444f358 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.570217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ad4d-7c41-425f-89f8-ed0f2444f358 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.570489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ad4d-7c41-425f-89f8-ed0f2444f358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.600655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dc06ad4d-7c41-425f-89f8-ed0f2444f358 None None] MFA Rules not processed for user `66fd41053c8845a3bb10e8899af2defb`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:19.601795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dc06ad4d-7c41-425f-89f8-ed0f2444f358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:19.660591 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 760/1535] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:19 2022] PUT /identity/v3/projects/7b1d101acb9549efa30ab0b0a25c32a4/users/3466eb5c5f4f465eb5b02fd297ec7298/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:19.661819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2c19a-8f29-4aa9-b5ae-b760f4270964 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.663769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6aa2c19a-8f29-4aa9-b5ae-b760f4270964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:19.673503 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 776/1536] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:19.675264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f83dcc-0d40-4bd3-8c9f-d471ca7bd860 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.675552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f83dcc-0d40-4bd3-8c9f-d471ca7bd860 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.675827 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f83dcc-0d40-4bd3-8c9f-d471ca7bd860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.683836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:19.709477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e3f83dcc-0d40-4bd3-8c9f-d471ca7bd860 None None] MFA Rules not processed for user `3466eb5c5f4f465eb5b02fd297ec7298`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:19.711019 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e3f83dcc-0d40-4bd3-8c9f-d471ca7bd860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:19.724714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:19.742765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2c19a-8f29-4aa9-b5ae-b760f4270964 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.747606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.748316 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.748571 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.748886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] PATH_INFO: `/v3/users/5fd2beb18ecc4202a5cdea346af2fd0c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.750049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd2beb18ecc4202a5cdea346af2fd0c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.752779 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.816205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 777/1537] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:19.817221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f83dcc-0d40-4bd3-8c9f-d471ca7bd860 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.818628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e3f83dcc-0d40-4bd3-8c9f-d471ca7bd860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:19.833674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f83dcc-0d40-4bd3-8c9f-d471ca7bd860 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:19.838480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a56cf-a9fa-4a2f-8830-f0dd02193970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:19.839381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a56cf-a9fa-4a2f-8830-f0dd02193970 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:19.842524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a56cf-a9fa-4a2f-8830-f0dd02193970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:19.842524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a56cf-a9fa-4a2f-8830-f0dd02193970 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:19.842524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a56cf-a9fa-4a2f-8830-f0dd02193970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:19.843696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a56cf-a9fa-4a2f-8830-f0dd02193970 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:19.952238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd2beb18ecc4202a5cdea346af2fd0c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:19.952905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd2beb18ecc4202a5cdea346af2fd0c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:19.962790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd2beb18ecc4202a5cdea346af2fd0c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:19.963558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd2beb18ecc4202a5cdea346af2fd0c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:19.969004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 778/1538] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:19 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:19.969004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a56cf-a9fa-4a2f-8830-f0dd02193970 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:19.969713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9a1a56cf-a9fa-4a2f-8830-f0dd02193970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:20.044224 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a56cf-a9fa-4a2f-8830-f0dd02193970 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.053034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540015b-3653-4b9b-b561-c834eb9a8b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.054044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540015b-3653-4b9b-b561-c834eb9a8b7d admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.054493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540015b-3653-4b9b-b561-c834eb9a8b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.054947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540015b-3653-4b9b-b561-c834eb9a8b7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.055872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540015b-3653-4b9b-b561-c834eb9a8b7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.058230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e3386b27-b808-482f-8d08-82883fc3e88a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd2beb18ecc4202a5cdea346af2fd0c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:20.062932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540015b-3653-4b9b-b561-c834eb9a8b7d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.089484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 761/1539] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:19 2022] DELETE /identity/v3/users/5fd2beb18ecc4202a5cdea346af2fd0c => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:20.091131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604170d-5d92-4cdf-938b-97a7a19259bd None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.091578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604170d-5d92-4cdf-938b-97a7a19259bd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.091862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604170d-5d92-4cdf-938b-97a7a19259bd None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.097379 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 762/1540] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 13:04:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:04:20.098572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604170d-5d92-4cdf-938b-97a7a19259bd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.103144 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9604170d-5d92-4cdf-938b-97a7a19259bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:20.145594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 779/1541] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:19 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:20.147014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d6be1-c21e-43cc-9525-707ac3214228 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.147326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d6be1-c21e-43cc-9525-707ac3214228 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.147583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d6be1-c21e-43cc-9525-707ac3214228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.184052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-195d6be1-c21e-43cc-9525-707ac3214228 None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:20.187831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-195d6be1-c21e-43cc-9525-707ac3214228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:20.232866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604170d-5d92-4cdf-938b-97a7a19259bd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.283248 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 780/1542] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 13:04:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:20.284116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d6be1-c21e-43cc-9525-707ac3214228 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.313040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:20.352054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e42ea9-309f-4c47-b887-963d247de0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.354447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e42ea9-309f-4c47-b887-963d247de0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.354799 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e42ea9-309f-4c47-b887-963d247de0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.355600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e42ea9-309f-4c47-b887-963d247de0d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.357230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e42ea9-309f-4c47-b887-963d247de0d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.365764 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e42ea9-309f-4c47-b887-963d247de0d5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.369200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d6be1-c21e-43cc-9525-707ac3214228 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.379051 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fe973-c567-4aa0-9a33-a7bb02830f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.379597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe973-c567-4aa0-9a33-a7bb02830f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.379881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe973-c567-4aa0-9a33-a7bb02830f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.380164 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe973-c567-4aa0-9a33-a7bb02830f01 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.383215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fe973-c567-4aa0-9a33-a7bb02830f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.389517 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fe973-c567-4aa0-9a33-a7bb02830f01 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.396723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:20.470273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 763/1543] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:20 2022] POST /identity/v3/users => generated 298 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:20.471138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e42ea9-309f-4c47-b887-963d247de0d5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.472853 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80e42ea9-309f-4c47-b887-963d247de0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:20.494951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e42ea9-309f-4c47-b887-963d247de0d5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.505934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 781/1544] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:20 2022] POST /identity/v3/users => generated 299 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:20.505934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fe973-c567-4aa0-9a33-a7bb02830f01 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.506216 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c86e95-8955-4d43-897a-922037385fdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.507779 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c86e95-8955-4d43-897a-922037385fdd service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.508069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c86e95-8955-4d43-897a-922037385fdd service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.508440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c86e95-8955-4d43-897a-922037385fdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.516024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fe973-c567-4aa0-9a33-a7bb02830f01 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.518361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c86e95-8955-4d43-897a-922037385fdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.519951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c86e95-8955-4d43-897a-922037385fdd service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.521428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f3813-1199-493c-ad72-b51b2ee69556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.522174 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f3813-1199-493c-ad72-b51b2ee69556 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.522522 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f3813-1199-493c-ad72-b51b2ee69556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.522866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f3813-1199-493c-ad72-b51b2ee69556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.523601 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f3813-1199-493c-ad72-b51b2ee69556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.525390 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f3813-1199-493c-ad72-b51b2ee69556 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.536211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 782/1545] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:20.536901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f3813-1199-493c-ad72-b51b2ee69556 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.557617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 764/1546] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:20.557617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c86e95-8955-4d43-897a-922037385fdd service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.557617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c86e95-8955-4d43-897a-922037385fdd service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.571638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f3813-1199-493c-ad72-b51b2ee69556 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.571754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ff0e-90be-434e-b5c5-4e6125f756c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.571972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ff0e-90be-434e-b5c5-4e6125f756c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.572391 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ff0e-90be-434e-b5c5-4e6125f756c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.572958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ff0e-90be-434e-b5c5-4e6125f756c9 admin admin] PATH_INFO: `/v3/projects/2ee56d1bead14af5b914648b55b672ba/users/5baff42ecf8741f5bcb72fdfaf4b3ca4/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.577223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6defd-551c-4247-8314-3c43279a5cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.577864 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6defd-551c-4247-8314-3c43279a5cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.578147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6defd-551c-4247-8314-3c43279a5cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.578428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6defd-551c-4247-8314-3c43279a5cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.579141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6defd-551c-4247-8314-3c43279a5cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.581329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6defd-551c-4247-8314-3c43279a5cf9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.595063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 783/1547] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:20.599250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6defd-551c-4247-8314-3c43279a5cf9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.599250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b2a6defd-551c-4247-8314-3c43279a5cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:20.601371 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103ff0e-90be-434e-b5c5-4e6125f756c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee56d1bead14af5b914648b55b672ba, user_id=5baff42ecf8741f5bcb72fdfaf4b3ca4, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.604368 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103ff0e-90be-434e-b5c5-4e6125f756c9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.615761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6defd-551c-4247-8314-3c43279a5cf9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.621352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.622016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.622302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.622572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] PATH_INFO: `/v3/projects/81ca09a0e852463f9a312f5043563dde` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.623743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ca09a0e852463f9a312f5043563dde)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.625697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.670363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 765/1548] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:20 2022] PUT /identity/v3/projects/2ee56d1bead14af5b914648b55b672ba/users/5baff42ecf8741f5bcb72fdfaf4b3ca4/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:20.671192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ff0e-90be-434e-b5c5-4e6125f756c9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.738048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ff0e-90be-434e-b5c5-4e6125f756c9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.754880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ca09a0e852463f9a312f5043563dde', 'request_id': 'req-597dc89b-190f-4de4-8d07-63c327b7e15e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:20.793890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6d3a6-24cd-447e-b7c1-4468e418028e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.794556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6d3a6-24cd-447e-b7c1-4468e418028e admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.794840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6d3a6-24cd-447e-b7c1-4468e418028e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.795113 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6d3a6-24cd-447e-b7c1-4468e418028e admin admin] PATH_INFO: `/v3/projects/96b0bdfbd18640298926e091dd87e34a/users/0ad52b2bf3af49f88c229f0b8b93be37/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.810152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6d3a6-24cd-447e-b7c1-4468e418028e admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b0bdfbd18640298926e091dd87e34a, user_id=0ad52b2bf3af49f88c229f0b8b93be37, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.812864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6d3a6-24cd-447e-b7c1-4468e418028e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.818572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ca09a0e852463f9a312f5043563dde', 'request_id': 'req-597dc89b-190f-4de4-8d07-63c327b7e15e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:20.825318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ca09a0e852463f9a312f5043563dde', 'request_id': 'req-597dc89b-190f-4de4-8d07-63c327b7e15e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:20.835534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] The token cache is being invalidate because project 81ca09a0e852463f9a312f5043563dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:20.835824 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:20.836464 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:20.838003 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 784/1549] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:20 2022] DELETE /identity/v3/projects/81ca09a0e852463f9a312f5043563dde => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:20.838762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.840048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:20.852720 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597dc89b-190f-4de4-8d07-63c327b7e15e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.857326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbf3e5-7d01-421c-ac01-45e260862b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.857908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbf3e5-7d01-421c-ac01-45e260862b2e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.858186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbf3e5-7d01-421c-ac01-45e260862b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.858454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbf3e5-7d01-421c-ac01-45e260862b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.859082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfbf3e5-7d01-421c-ac01-45e260862b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.861968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfbf3e5-7d01-421c-ac01-45e260862b2e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.871669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 766/1550] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:20 2022] PUT /identity/v3/projects/96b0bdfbd18640298926e091dd87e34a/users/0ad52b2bf3af49f88c229f0b8b93be37/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:20.873229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9404dc3-0499-4364-8f0b-32bc3a655b0d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.873672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9404dc3-0499-4364-8f0b-32bc3a655b0d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.874052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9404dc3-0499-4364-8f0b-32bc3a655b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.878723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 785/1551] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:20.878723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbf3e5-7d01-421c-ac01-45e260862b2e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.878723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9dfbf3e5-7d01-421c-ac01-45e260862b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:20.890963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbf3e5-7d01-421c-ac01-45e260862b2e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.896582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b9404dc3-0499-4364-8f0b-32bc3a655b0d None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:20.897846 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b9404dc3-0499-4364-8f0b-32bc3a655b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:20.940415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:20.953321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529c9ef-c43d-4398-b04f-7f30a9f59c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.954917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529c9ef-c43d-4398-b04f-7f30a9f59c6e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.954917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529c9ef-c43d-4398-b04f-7f30a9f59c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.955166 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529c9ef-c43d-4398-b04f-7f30a9f59c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.955761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529c9ef-c43d-4398-b04f-7f30a9f59c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.957404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529c9ef-c43d-4398-b04f-7f30a9f59c6e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.965881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 767/1552] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:20.970229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9404dc3-0499-4364-8f0b-32bc3a655b0d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.970660 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 786/1553] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:20.974290 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529c9ef-c43d-4398-b04f-7f30a9f59c6e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:20.981263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9404dc3-0499-4364-8f0b-32bc3a655b0d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.984562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529c9ef-c43d-4398-b04f-7f30a9f59c6e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:20.988459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dc554-549c-490a-8dce-16cc88687f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.988459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc554-549c-490a-8dce-16cc88687f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.988459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc554-549c-490a-8dce-16cc88687f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.988459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc554-549c-490a-8dce-16cc88687f1a admin admin] PATH_INFO: `/v3/projects/2ee56d1bead14af5b914648b55b672ba/users/5baff42ecf8741f5bcb72fdfaf4b3ca4/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.990438 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dc554-549c-490a-8dce-16cc88687f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee56d1bead14af5b914648b55b672ba, user_id=5baff42ecf8741f5bcb72fdfaf4b3ca4, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.991525 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c8413-4ec0-41eb-855a-851097bd6ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:20.992249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c8413-4ec0-41eb-855a-851097bd6ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:20.992533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c8413-4ec0-41eb-855a-851097bd6ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:20.992812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c8413-4ec0-41eb-855a-851097bd6ec8 admin admin] PATH_INFO: `/v3/projects/96b0bdfbd18640298926e091dd87e34a/users/0ad52b2bf3af49f88c229f0b8b93be37/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:20.993302 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dc554-549c-490a-8dce-16cc88687f1a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:20.995456 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c8413-4ec0-41eb-855a-851097bd6ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b0bdfbd18640298926e091dd87e34a, user_id=0ad52b2bf3af49f88c229f0b8b93be37, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:20.998369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c8413-4ec0-41eb-855a-851097bd6ec8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:21.004849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 768/1554] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:20 2022] PUT /identity/v3/projects/2ee56d1bead14af5b914648b55b672ba/users/5baff42ecf8741f5bcb72fdfaf4b3ca4/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:21.006164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887669bb-9703-4d94-89d8-0e34723985c3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.006448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887669bb-9703-4d94-89d8-0e34723985c3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.007248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887669bb-9703-4d94-89d8-0e34723985c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.017658 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:21.021507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 787/1555] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:20 2022] PUT /identity/v3/projects/96b0bdfbd18640298926e091dd87e34a/users/0ad52b2bf3af49f88c229f0b8b93be37/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:21.031357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7211739-4ad6-41a7-8fc4-d88fae84d60f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.031357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7211739-4ad6-41a7-8fc4-d88fae84d60f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.031357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7211739-4ad6-41a7-8fc4-d88fae84d60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.032350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-887669bb-9703-4d94-89d8-0e34723985c3 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:21.032350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-887669bb-9703-4d94-89d8-0e34723985c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:21.058016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e7211739-4ad6-41a7-8fc4-d88fae84d60f None None] MFA Rules not processed for user `5baff42ecf8741f5bcb72fdfaf4b3ca4`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:21.059242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e7211739-4ad6-41a7-8fc4-d88fae84d60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:21.099498 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 769/1556] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:21.103072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd7b5-ff61-4a61-a707-c4cf39a250e3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.103543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd7b5-ff61-4a61-a707-c4cf39a250e3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.104203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd7b5-ff61-4a61-a707-c4cf39a250e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.133821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-854dd7b5-ff61-4a61-a707-c4cf39a250e3 None None] MFA Rules not processed for user `0ad52b2bf3af49f88c229f0b8b93be37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:21.134135 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 788/1557] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:21.135108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-854dd7b5-ff61-4a61-a707-c4cf39a250e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:21.135204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7211739-4ad6-41a7-8fc4-d88fae84d60f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:21.146372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7211739-4ad6-41a7-8fc4-d88fae84d60f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:21.151075 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:21.151751 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.152104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.152450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.153163 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:21.153646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:21.155425 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:21.167436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 789/1558] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:21.173121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:21.173121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:21.208571 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 770/1559] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:21.209374 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd7b5-ff61-4a61-a707-c4cf39a250e3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:21.221159 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd7b5-ff61-4a61-a707-c4cf39a250e3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:21.225933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82a52b-08aa-4a7a-8e61-5909332a55b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:21.226546 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a52b-08aa-4a7a-8e61-5909332a55b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.226826 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a52b-08aa-4a7a-8e61-5909332a55b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.227124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a52b-08aa-4a7a-8e61-5909332a55b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.227768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82a52b-08aa-4a7a-8e61-5909332a55b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:21.230283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82a52b-08aa-4a7a-8e61-5909332a55b6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:21.230411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc181c5e-9a04-4a66-85d5-d81257baf6b7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:21.233917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83bfca-25a9-417c-b746-ce3320696f37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:21.234534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83bfca-25a9-417c-b746-ce3320696f37 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.234803 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83bfca-25a9-417c-b746-ce3320696f37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.235079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83bfca-25a9-417c-b746-ce3320696f37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.237648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1c83bfca-25a9-417c-b746-ce3320696f37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:21.294787 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 771/1560] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:21 2022] POST /identity/v3/projects => generated 387 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:21.295767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82a52b-08aa-4a7a-8e61-5909332a55b6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:21.300862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ea82a52b-08aa-4a7a-8e61-5909332a55b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:21.318198 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83bfca-25a9-417c-b746-ce3320696f37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:21.322027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83bfca-25a9-417c-b746-ce3320696f37 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:21.326750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:21.326949 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:21.337864 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 790/1561] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:21.339051 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83bfca-25a9-417c-b746-ce3320696f37 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:21.353641 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83bfca-25a9-417c-b746-ce3320696f37 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:21.438636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82a52b-08aa-4a7a-8e61-5909332a55b6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:21.450589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d958a7-c6fa-4507-bec6-e1fce2236527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:21.451239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d958a7-c6fa-4507-bec6-e1fce2236527 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.451584 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d958a7-c6fa-4507-bec6-e1fce2236527 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.451801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d958a7-c6fa-4507-bec6-e1fce2236527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.461063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e7eb9-913f-42b1-b827-3eb75d43cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:21.461063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e7eb9-913f-42b1-b827-3eb75d43cb65 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.461063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e7eb9-913f-42b1-b827-3eb75d43cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.461063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e7eb9-913f-42b1-b827-3eb75d43cb65 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.461559 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e7eb9-913f-42b1-b827-3eb75d43cb65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:21.462604 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e7eb9-913f-42b1-b827-3eb75d43cb65 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:21.614162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 791/1562] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:21 2022] POST /identity/v3/users => generated 294 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:21.616293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d958a7-c6fa-4507-bec6-e1fce2236527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:21.619352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e7eb9-913f-42b1-b827-3eb75d43cb65 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:21.620146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d958a7-c6fa-4507-bec6-e1fce2236527 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:21.631646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e7eb9-913f-42b1-b827-3eb75d43cb65 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:21.632003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 772/1563] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:21.640404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076dde6-9c82-4421-a41e-dfa9832b6fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:21.641226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076dde6-9c82-4421-a41e-dfa9832b6fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.644910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076dde6-9c82-4421-a41e-dfa9832b6fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.644910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076dde6-9c82-4421-a41e-dfa9832b6fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.644910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076dde6-9c82-4421-a41e-dfa9832b6fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:21.646324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076dde6-9c82-4421-a41e-dfa9832b6fa5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:21.663142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d958a7-c6fa-4507-bec6-e1fce2236527 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:21.670164 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 792/1564] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:21.678543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d958a7-c6fa-4507-bec6-e1fce2236527 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:21.684895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97e1e8-edfe-4c35-bfcb-14d474cd7fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:21.686487 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e1e8-edfe-4c35-bfcb-14d474cd7fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.687038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e1e8-edfe-4c35-bfcb-14d474cd7fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.688908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e1e8-edfe-4c35-bfcb-14d474cd7fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.690363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97e1e8-edfe-4c35-bfcb-14d474cd7fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:21.693821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97e1e8-edfe-4c35-bfcb-14d474cd7fc9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:21.715414 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 773/1565] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:21.719475 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076dde6-9c82-4421-a41e-dfa9832b6fa5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:21.735889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076dde6-9c82-4421-a41e-dfa9832b6fa5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:21.741374 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08cf42-45e4-42a0-9569-cba75ef35e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:21.744737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08cf42-45e4-42a0-9569-cba75ef35e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.744737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08cf42-45e4-42a0-9569-cba75ef35e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.744737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08cf42-45e4-42a0-9569-cba75ef35e58 admin admin] PATH_INFO: `/v3/projects/a9a6240e83f14215a04fecee2306f7f1/users/ad3e6b90fa584f2189e520475d6a6ea9/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.772416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08cf42-45e4-42a0-9569-cba75ef35e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a6240e83f14215a04fecee2306f7f1, user_id=ad3e6b90fa584f2189e520475d6a6ea9, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:21.776319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08cf42-45e4-42a0-9569-cba75ef35e58 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:21.897224 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 793/1566] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:21 2022] PUT /identity/v3/projects/a9a6240e83f14215a04fecee2306f7f1/users/ad3e6b90fa584f2189e520475d6a6ea9/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:21.905653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfc196-ce1f-4230-b671-9eaf825f98f4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:21.905945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfc196-ce1f-4230-b671-9eaf825f98f4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:21.906256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfc196-ce1f-4230-b671-9eaf825f98f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:21.947165 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0dbfc196-ce1f-4230-b671-9eaf825f98f4 None None] MFA Rules not processed for user `ad3e6b90fa584f2189e520475d6a6ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:21.948533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0dbfc196-ce1f-4230-b671-9eaf825f98f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:22.027340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 774/1567] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:22.032860 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08cf42-45e4-42a0-9569-cba75ef35e58 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:22.117961 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08cf42-45e4-42a0-9569-cba75ef35e58 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:22.122705 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da6d58-5f4f-4afc-b347-64f4f25a4df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:22.123468 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da6d58-5f4f-4afc-b347-64f4f25a4df4 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:22.123875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da6d58-5f4f-4afc-b347-64f4f25a4df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:22.124282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da6d58-5f4f-4afc-b347-64f4f25a4df4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:22.125021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da6d58-5f4f-4afc-b347-64f4f25a4df4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:22.127531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da6d58-5f4f-4afc-b347-64f4f25a4df4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:22.240104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 794/1568] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:22 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:22.243793 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfc196-ce1f-4230-b671-9eaf825f98f4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:22.260806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfc196-ce1f-4230-b671-9eaf825f98f4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:22.312404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:22.332622 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e9d53-8db5-4d94-be6e-5c1e12e50d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:22.333349 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e9d53-8db5-4d94-be6e-5c1e12e50d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:22.333770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e9d53-8db5-4d94-be6e-5c1e12e50d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:22.334054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e9d53-8db5-4d94-be6e-5c1e12e50d1a admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:22.334757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e9d53-8db5-4d94-be6e-5c1e12e50d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:22.336733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e9d53-8db5-4d94-be6e-5c1e12e50d1a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:22.429449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 775/1569] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:22 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:22.430753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da6d58-5f4f-4afc-b347-64f4f25a4df4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:22.441252 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da6d58-5f4f-4afc-b347-64f4f25a4df4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:22.445789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a85490-bfe2-4467-acfb-a8f26a98c86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:22.446389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a85490-bfe2-4467-acfb-a8f26a98c86b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:22.446665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a85490-bfe2-4467-acfb-a8f26a98c86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:22.446934 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a85490-bfe2-4467-acfb-a8f26a98c86b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:22.447571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a85490-bfe2-4467-acfb-a8f26a98c86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:22.449164 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a85490-bfe2-4467-acfb-a8f26a98c86b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:22.459709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 795/1570] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:22.463744 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e9d53-8db5-4d94-be6e-5c1e12e50d1a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:22.474716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e9d53-8db5-4d94-be6e-5c1e12e50d1a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:22.479418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317bdb6-4293-4962-9b14-f04bcf3c47ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:22.480021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317bdb6-4293-4962-9b14-f04bcf3c47ea admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:22.480301 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317bdb6-4293-4962-9b14-f04bcf3c47ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:22.480576 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317bdb6-4293-4962-9b14-f04bcf3c47ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:22.481266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317bdb6-4293-4962-9b14-f04bcf3c47ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:22.482879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317bdb6-4293-4962-9b14-f04bcf3c47ea admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:22.493501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 776/1571] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:22.497832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a85490-bfe2-4467-acfb-a8f26a98c86b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:22.507637 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a85490-bfe2-4467-acfb-a8f26a98c86b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:22.512560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda56a9a-8195-4020-bbb1-224128a0869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:22.513220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda56a9a-8195-4020-bbb1-224128a0869b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:22.513505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda56a9a-8195-4020-bbb1-224128a0869b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:22.513783 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda56a9a-8195-4020-bbb1-224128a0869b admin admin] PATH_INFO: `/v3/projects/e4868f3cfe85453c8bfafde9cd8aba8d/users/a31d6c721707462ebfeee8c40dea7d01/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:22.528738 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda56a9a-8195-4020-bbb1-224128a0869b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4868f3cfe85453c8bfafde9cd8aba8d, user_id=a31d6c721707462ebfeee8c40dea7d01, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:22.531482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda56a9a-8195-4020-bbb1-224128a0869b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:22.603084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 796/1572] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:22 2022] PUT /identity/v3/projects/e4868f3cfe85453c8bfafde9cd8aba8d/users/a31d6c721707462ebfeee8c40dea7d01/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:22.620478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1a1d7-7c8e-49f1-848b-4c72403bd8c7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:22.620842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1a1d7-7c8e-49f1-848b-4c72403bd8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:22.621129 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1a1d7-7c8e-49f1-848b-4c72403bd8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:22.662252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d5b1a1d7-7c8e-49f1-848b-4c72403bd8c7 None None] MFA Rules not processed for user `a31d6c721707462ebfeee8c40dea7d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:22.663627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d5b1a1d7-7c8e-49f1-848b-4c72403bd8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:22.749468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 777/1573] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:22.754058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda56a9a-8195-4020-bbb1-224128a0869b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:22.760350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:22.849947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda56a9a-8195-4020-bbb1-224128a0869b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:22.854729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d6a7e-aba1-4cb3-a5af-8f33505b1ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:22.855335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d6a7e-aba1-4cb3-a5af-8f33505b1ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:22.855618 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d6a7e-aba1-4cb3-a5af-8f33505b1ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:22.855896 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d6a7e-aba1-4cb3-a5af-8f33505b1ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:22.856541 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d6a7e-aba1-4cb3-a5af-8f33505b1ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:22.858521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d6a7e-aba1-4cb3-a5af-8f33505b1ec7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:22.936915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1a1d7-7c8e-49f1-848b-4c72403bd8c7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:22.948726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1a1d7-7c8e-49f1-848b-4c72403bd8c7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:22.956828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 797/1574] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:22 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:22.956957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:22.956957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:22.956957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:22.956957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] PATH_INFO: `/v3/users/3466eb5c5f4f465eb5b02fd297ec7298` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:22.957365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3466eb5c5f4f465eb5b02fd297ec7298)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:22.958158 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d6a7e-aba1-4cb3-a5af-8f33505b1ec7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:22.958687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:23.035970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d6a7e-aba1-4cb3-a5af-8f33505b1ec7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:23.040860 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fa1a3-cff1-494c-8c82-ec1a55d0fac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:23.041590 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fa1a3-cff1-494c-8c82-ec1a55d0fac6 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:23.041925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fa1a3-cff1-494c-8c82-ec1a55d0fac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:23.042303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fa1a3-cff1-494c-8c82-ec1a55d0fac6 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:23.043098 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fa1a3-cff1-494c-8c82-ec1a55d0fac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:23.045133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fa1a3-cff1-494c-8c82-ec1a55d0fac6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:23.167648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 798/1575] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:22 2022] POST /identity/v3/users => generated 300 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:23.171706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fa1a3-cff1-494c-8c82-ec1a55d0fac6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:23.183133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fa1a3-cff1-494c-8c82-ec1a55d0fac6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:23.189409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0283bf6-4e50-4a77-b5ab-f16e1247caf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:23.190096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0283bf6-4e50-4a77-b5ab-f16e1247caf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:23.190807 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0283bf6-4e50-4a77-b5ab-f16e1247caf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:23.190807 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0283bf6-4e50-4a77-b5ab-f16e1247caf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:23.191328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0283bf6-4e50-4a77-b5ab-f16e1247caf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:23.193155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0283bf6-4e50-4a77-b5ab-f16e1247caf6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:23.205734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 799/1576] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:23.209486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0283bf6-4e50-4a77-b5ab-f16e1247caf6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:23.220926 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0283bf6-4e50-4a77-b5ab-f16e1247caf6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:23.229054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e41499-8653-4b69-b323-a7eb734e4e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:23.229758 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e41499-8653-4b69-b323-a7eb734e4e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:23.230044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e41499-8653-4b69-b323-a7eb734e4e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:23.230322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e41499-8653-4b69-b323-a7eb734e4e13 admin admin] PATH_INFO: `/v3/projects/967c2861dac34a528b80e9db018ace37/users/56b7f292b5ca492abc7c4b627eed88ad/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:23.246406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e41499-8653-4b69-b323-a7eb734e4e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=967c2861dac34a528b80e9db018ace37, user_id=56b7f292b5ca492abc7c4b627eed88ad, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:23.249083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e41499-8653-4b69-b323-a7eb734e4e13 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:23.300569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3466eb5c5f4f465eb5b02fd297ec7298'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:23.301373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3466eb5c5f4f465eb5b02fd297ec7298'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:23.309044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3466eb5c5f4f465eb5b02fd297ec7298'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:23.309827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3466eb5c5f4f465eb5b02fd297ec7298'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:23.375085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 800/1577] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:23 2022] PUT /identity/v3/projects/967c2861dac34a528b80e9db018ace37/users/56b7f292b5ca492abc7c4b627eed88ad/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:23.379151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e41499-8653-4b69-b323-a7eb734e4e13 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:23.380668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-28e41499-8653-4b69-b323-a7eb734e4e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:23.401478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f4d31e-6768-4a92-a1a8-04b1cbcaa7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3466eb5c5f4f465eb5b02fd297ec7298'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:23.428335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 778/1578] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:22 2022] DELETE /identity/v3/users/3466eb5c5f4f465eb5b02fd297ec7298 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:23.436331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3be9d-e8d3-4b3a-a734-2f77bae6d761 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:23.436743 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3be9d-e8d3-4b3a-a734-2f77bae6d761 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:23.437221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3be9d-e8d3-4b3a-a734-2f77bae6d761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:23.449471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:23.480769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5bd3be9d-e8d3-4b3a-a734-2f77bae6d761 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:23.480769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5bd3be9d-e8d3-4b3a-a734-2f77bae6d761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:23.493217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:23.493406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e41499-8653-4b69-b323-a7eb734e4e13 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:23.571883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 779/1579] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:23.587738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3be9d-e8d3-4b3a-a734-2f77bae6d761 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:23.590696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5bd3be9d-e8d3-4b3a-a734-2f77bae6d761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:23.648322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f9bcb-99c8-4ab7-8f79-b15020595531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:23.651196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f9bcb-99c8-4ab7-8f79-b15020595531 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:23.653210 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f9bcb-99c8-4ab7-8f79-b15020595531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:23.658508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f9bcb-99c8-4ab7-8f79-b15020595531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:23.659459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f9bcb-99c8-4ab7-8f79-b15020595531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:23.661710 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f9bcb-99c8-4ab7-8f79-b15020595531 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:23.682633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 801/1580] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:23.688685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f9bcb-99c8-4ab7-8f79-b15020595531 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:23.701354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f9bcb-99c8-4ab7-8f79-b15020595531 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:23.708405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ed242-1b87-4ccb-8e5e-a49b4e85b3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:23.709850 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ed242-1b87-4ccb-8e5e-a49b4e85b3f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:23.710314 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ed242-1b87-4ccb-8e5e-a49b4e85b3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:23.710767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ed242-1b87-4ccb-8e5e-a49b4e85b3f2 admin admin] PATH_INFO: `/v3/projects/967c2861dac34a528b80e9db018ace37/users/56b7f292b5ca492abc7c4b627eed88ad/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:23.714114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ed242-1b87-4ccb-8e5e-a49b4e85b3f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=967c2861dac34a528b80e9db018ace37, user_id=56b7f292b5ca492abc7c4b627eed88ad, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:23.718737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ed242-1b87-4ccb-8e5e-a49b4e85b3f2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:23.734082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 802/1581] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:23 2022] PUT /identity/v3/projects/967c2861dac34a528b80e9db018ace37/users/56b7f292b5ca492abc7c4b627eed88ad/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:23.740536 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3be9d-e8d3-4b3a-a734-2f77bae6d761 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:23.744862 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4af0e-3fc6-4a78-a952-fb0a86828abb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:23.745930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4af0e-3fc6-4a78-a952-fb0a86828abb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:23.746400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4af0e-3fc6-4a78-a952-fb0a86828abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:23.811259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-65a4af0e-3fc6-4a78-a952-fb0a86828abb None None] MFA Rules not processed for user `56b7f292b5ca492abc7c4b627eed88ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:23.812465 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-65a4af0e-3fc6-4a78-a952-fb0a86828abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:23.848941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bdc35-25d2-4cf1-b76a-4dab23627b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:23.849615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bdc35-25d2-4cf1-b76a-4dab23627b64 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:23.849902 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bdc35-25d2-4cf1-b76a-4dab23627b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:23.850237 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bdc35-25d2-4cf1-b76a-4dab23627b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:23.926447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 803/1582] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:23.946173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4af0e-3fc6-4a78-a952-fb0a86828abb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:23.948549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-65a4af0e-3fc6-4a78-a952-fb0a86828abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:24.018948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bdc35-25d2-4cf1-b76a-4dab23627b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:24.021465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bdc35-25d2-4cf1-b76a-4dab23627b64 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:24.035596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 780/1583] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:24.057893 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4af0e-3fc6-4a78-a952-fb0a86828abb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:24.062268 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129b57d-4fdd-4f69-acd8-aa3423976c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:24.062957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129b57d-4fdd-4f69-acd8-aa3423976c14 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:24.064894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129b57d-4fdd-4f69-acd8-aa3423976c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:24.065553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129b57d-4fdd-4f69-acd8-aa3423976c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:24.076688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129b57d-4fdd-4f69-acd8-aa3423976c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:24.078392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129b57d-4fdd-4f69-acd8-aa3423976c14 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:24.084791 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:24.090670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 804/1584] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:24.662311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bdc35-25d2-4cf1-b76a-4dab23627b64 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:24.664060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e21bdc35-25d2-4cf1-b76a-4dab23627b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:24.680806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bdc35-25d2-4cf1-b76a-4dab23627b64 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:24.695234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:24.696647 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:24.705626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:24.705978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] PATH_INFO: `/v3/projects/7b1d101acb9549efa30ab0b0a25c32a4` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:24.707296 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1d101acb9549efa30ab0b0a25c32a4)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:24.709416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:24.737036 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:24.865398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1d101acb9549efa30ab0b0a25c32a4', 'request_id': 'req-2f624b7c-19ba-460f-98bb-11b4c4586b89'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:24.871858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1d101acb9549efa30ab0b0a25c32a4', 'request_id': 'req-2f624b7c-19ba-460f-98bb-11b4c4586b89'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:24.961282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1d101acb9549efa30ab0b0a25c32a4', 'request_id': 'req-2f624b7c-19ba-460f-98bb-11b4c4586b89'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:24.976182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] The token cache is being invalidate because project 7b1d101acb9549efa30ab0b0a25c32a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:24.976182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:24.976182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:24.976182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 781/1585] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:24 2022] DELETE /identity/v3/projects/7b1d101acb9549efa30ab0b0a25c32a4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:24.985942 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129b57d-4fdd-4f69-acd8-aa3423976c14 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:24.999580 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129b57d-4fdd-4f69-acd8-aa3423976c14 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:25.059988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:25.060631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:25.060912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:25.061209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] PATH_INFO: `/v3/users/0ad52b2bf3af49f88c229f0b8b93be37` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:25.062360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad52b2bf3af49f88c229f0b8b93be37)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:25.064996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:25.321099 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:25.322791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:25.324287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad52b2bf3af49f88c229f0b8b93be37'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:25.334667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad52b2bf3af49f88c229f0b8b93be37'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:25.335638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad52b2bf3af49f88c229f0b8b93be37'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:25.336299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad52b2bf3af49f88c229f0b8b93be37'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:25.418725 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad52b2bf3af49f88c229f0b8b93be37'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:25.459729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 805/1586] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:24 2022] DELETE /identity/v3/users/0ad52b2bf3af49f88c229f0b8b93be37 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:25.567109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f624b7c-19ba-460f-98bb-11b4c4586b89 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:25.626593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb98c4-456c-4f83-b813-322132803e2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:25.627220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb98c4-456c-4f83-b813-322132803e2c service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:25.627508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb98c4-456c-4f83-b813-322132803e2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:25.627787 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb98c4-456c-4f83-b813-322132803e2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:25.630451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d0bb98c4-456c-4f83-b813-322132803e2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:25.702203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb98c4-456c-4f83-b813-322132803e2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:25.704342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb98c4-456c-4f83-b813-322132803e2c service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:25.729593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 782/1587] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:26.106031 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:26.107589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:26.211847 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1f950-51c5-423f-aebd-8b2aa30fc604 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:26.218748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:26.220777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:26.221486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:26.221975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] PATH_INFO: `/v3/projects/96b0bdfbd18640298926e091dd87e34a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:26.223624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b0bdfbd18640298926e091dd87e34a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:26.227378 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:26.295212 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:26.411805 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b0bdfbd18640298926e091dd87e34a', 'request_id': 'req-ed0b2cac-5983-4c69-92df-b869a8799253'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:26.484733 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b0bdfbd18640298926e091dd87e34a', 'request_id': 'req-ed0b2cac-5983-4c69-92df-b869a8799253'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:26.515723 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b0bdfbd18640298926e091dd87e34a', 'request_id': 'req-ed0b2cac-5983-4c69-92df-b869a8799253'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:26.554364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] The token cache is being invalidate because project 96b0bdfbd18640298926e091dd87e34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:26.554697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:26.557785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ed0b2cac-5983-4c69-92df-b869a8799253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:26.559356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 806/1588] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:26 2022] DELETE /identity/v3/projects/96b0bdfbd18640298926e091dd87e34a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:26.565173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6049e-3ea8-4e04-93d6-ee74076bc9a9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:26.565624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6049e-3ea8-4e04-93d6-ee74076bc9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:26.566005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6049e-3ea8-4e04-93d6-ee74076bc9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:26.587613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e1d6049e-3ea8-4e04-93d6-ee74076bc9a9 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:26.589085 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e1d6049e-3ea8-4e04-93d6-ee74076bc9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:26.605152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:26.675027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 783/1589] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:26.675693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba67e3-8d97-42b7-9c89-3dd1b2de3cc1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:26.676080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba67e3-8d97-42b7-9c89-3dd1b2de3cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:26.677102 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba67e3-8d97-42b7-9c89-3dd1b2de3cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:26.732571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-43ba67e3-8d97-42b7-9c89-3dd1b2de3cc1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:26.734922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-43ba67e3-8d97-42b7-9c89-3dd1b2de3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:26.751741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 807/1590] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:26.757884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6049e-3ea8-4e04-93d6-ee74076bc9a9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:26.776840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6049e-3ea8-4e04-93d6-ee74076bc9a9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:26.781988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad66f7-4b4e-4f7e-8cfe-227668deb4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:26.783025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad66f7-4b4e-4f7e-8cfe-227668deb4ae admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:26.783522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad66f7-4b4e-4f7e-8cfe-227668deb4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:26.784067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad66f7-4b4e-4f7e-8cfe-227668deb4ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:26.784932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad66f7-4b4e-4f7e-8cfe-227668deb4ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:26.785439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad66f7-4b4e-4f7e-8cfe-227668deb4ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:26.790037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad66f7-4b4e-4f7e-8cfe-227668deb4ae admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:26.814216 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 784/1591] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:26.816258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba67e3-8d97-42b7-9c89-3dd1b2de3cc1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:26.826110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba67e3-8d97-42b7-9c89-3dd1b2de3cc1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:26.834092 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa30caa-3668-467b-8902-de863c067a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:26.839613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa30caa-3668-467b-8902-de863c067a8a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:26.839613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa30caa-3668-467b-8902-de863c067a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:26.839613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa30caa-3668-467b-8902-de863c067a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:26.839613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa30caa-3668-467b-8902-de863c067a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:26.839613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa30caa-3668-467b-8902-de863c067a8a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:27.012359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 808/1592] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:26 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:27.016158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad66f7-4b4e-4f7e-8cfe-227668deb4ae admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:27.027950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad66f7-4b4e-4f7e-8cfe-227668deb4ae admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:27.100305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4647c0-fc97-4a50-bda4-dc9f2a2aae30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:27.100921 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4647c0-fc97-4a50-bda4-dc9f2a2aae30 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:27.101217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4647c0-fc97-4a50-bda4-dc9f2a2aae30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:27.101519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4647c0-fc97-4a50-bda4-dc9f2a2aae30 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:27.102155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4647c0-fc97-4a50-bda4-dc9f2a2aae30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:27.104076 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4647c0-fc97-4a50-bda4-dc9f2a2aae30 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:27.208893 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 785/1593] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:27 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:27.212941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa30caa-3668-467b-8902-de863c067a8a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:27.223957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa30caa-3668-467b-8902-de863c067a8a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:27.228958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868e53f-00cc-416a-9129-cb173f1eae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:27.229662 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868e53f-00cc-416a-9129-cb173f1eae83 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:27.229882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868e53f-00cc-416a-9129-cb173f1eae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:27.230156 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868e53f-00cc-416a-9129-cb173f1eae83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:27.230806 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868e53f-00cc-416a-9129-cb173f1eae83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:27.232752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868e53f-00cc-416a-9129-cb173f1eae83 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:27.242788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 809/1594] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:27.246831 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4647c0-fc97-4a50-bda4-dc9f2a2aae30 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:27.256869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4647c0-fc97-4a50-bda4-dc9f2a2aae30 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:27.261573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf6a0e-a64a-4b33-bd24-2d8df501c1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:27.262188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6a0e-a64a-4b33-bd24-2d8df501c1fe admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:27.262482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6a0e-a64a-4b33-bd24-2d8df501c1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:27.262745 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6a0e-a64a-4b33-bd24-2d8df501c1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:27.263416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf6a0e-a64a-4b33-bd24-2d8df501c1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:27.265002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf6a0e-a64a-4b33-bd24-2d8df501c1fe admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:27.275564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 786/1595] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:27.279734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868e53f-00cc-416a-9129-cb173f1eae83 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:27.291186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868e53f-00cc-416a-9129-cb173f1eae83 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:27.296231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767b6a8-4295-4aa4-888c-993078967e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:27.296588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767b6a8-4295-4aa4-888c-993078967e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:27.296854 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767b6a8-4295-4aa4-888c-993078967e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:27.297136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767b6a8-4295-4aa4-888c-993078967e08 admin admin] PATH_INFO: `/v3/projects/c6d8449ffd4e4727b7b20eda3f02c287/users/ba9e8dbc77fc4772a127391a70697909/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:27.316855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767b6a8-4295-4aa4-888c-993078967e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d8449ffd4e4727b7b20eda3f02c287, user_id=ba9e8dbc77fc4772a127391a70697909, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:27.319596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767b6a8-4295-4aa4-888c-993078967e08 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:27.447483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 810/1596] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:27 2022] PUT /identity/v3/projects/c6d8449ffd4e4727b7b20eda3f02c287/users/ba9e8dbc77fc4772a127391a70697909/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:27.458423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a661e6a-76ef-4ffd-b753-0dc49d067859 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:27.458820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a661e6a-76ef-4ffd-b753-0dc49d067859 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:27.459188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a661e6a-76ef-4ffd-b753-0dc49d067859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:27.491233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2a661e6a-76ef-4ffd-b753-0dc49d067859 None None] MFA Rules not processed for user `ba9e8dbc77fc4772a127391a70697909`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:27.493819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2a661e6a-76ef-4ffd-b753-0dc49d067859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:27.577026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 787/1597] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:27.578055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767b6a8-4295-4aa4-888c-993078967e08 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:27.647107 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767b6a8-4295-4aa4-888c-993078967e08 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:27.652159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec82b3e-7832-467b-920a-0bbfc23dbeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:27.652889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec82b3e-7832-467b-920a-0bbfc23dbeeb admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:27.653262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec82b3e-7832-467b-920a-0bbfc23dbeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:27.653612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec82b3e-7832-467b-920a-0bbfc23dbeeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:27.654339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec82b3e-7832-467b-920a-0bbfc23dbeeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:27.656342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec82b3e-7832-467b-920a-0bbfc23dbeeb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:27.799128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 811/1598] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:27 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:27.803277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a661e6a-76ef-4ffd-b753-0dc49d067859 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:27.815776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a661e6a-76ef-4ffd-b753-0dc49d067859 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:27.829203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:27.877526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4baac-4ffe-4159-b496-4c0c9a11c0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:27.878176 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4baac-4ffe-4159-b496-4c0c9a11c0fa admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:27.878455 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4baac-4ffe-4159-b496-4c0c9a11c0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:27.878726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4baac-4ffe-4159-b496-4c0c9a11c0fa admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:27.879371 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4baac-4ffe-4159-b496-4c0c9a11c0fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:27.881315 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4baac-4ffe-4159-b496-4c0c9a11c0fa admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:27.998169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 788/1599] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:27 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:28.002000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec82b3e-7832-467b-920a-0bbfc23dbeeb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:28.026500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec82b3e-7832-467b-920a-0bbfc23dbeeb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:28.040859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc413f-23f9-44b2-8a0e-62d21ff6d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:28.045599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc413f-23f9-44b2-8a0e-62d21ff6d4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:28.045599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc413f-23f9-44b2-8a0e-62d21ff6d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:28.045599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc413f-23f9-44b2-8a0e-62d21ff6d4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:28.045599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc413f-23f9-44b2-8a0e-62d21ff6d4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:28.045599 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc413f-23f9-44b2-8a0e-62d21ff6d4d6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:28.062301 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4baac-4ffe-4159-b496-4c0c9a11c0fa admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:28.069454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 812/1600] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:28.073615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4baac-4ffe-4159-b496-4c0c9a11c0fa admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:28.078670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87052dd-fa08-4bcc-ad81-49134e41a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:28.080071 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87052dd-fa08-4bcc-ad81-49134e41a9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:28.080459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87052dd-fa08-4bcc-ad81-49134e41a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:28.080842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87052dd-fa08-4bcc-ad81-49134e41a9f3 admin admin] PATH_INFO: `/v3/projects/42426cccf1e5436d933f6f15321c9ed2/users/7fe4f01497904b1899f13aafbbbaf7d5/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:28.099886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87052dd-fa08-4bcc-ad81-49134e41a9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42426cccf1e5436d933f6f15321c9ed2, user_id=7fe4f01497904b1899f13aafbbbaf7d5, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:28.103405 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87052dd-fa08-4bcc-ad81-49134e41a9f3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:28.191291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 789/1601] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:28 2022] PUT /identity/v3/projects/42426cccf1e5436d933f6f15321c9ed2/users/7fe4f01497904b1899f13aafbbbaf7d5/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:28.196360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc413f-23f9-44b2-8a0e-62d21ff6d4d6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:28.209900 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc413f-23f9-44b2-8a0e-62d21ff6d4d6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:28.230106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:28.278519 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ad430-0f7f-4d32-9034-d79902ba77a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:28.279216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ad430-0f7f-4d32-9034-d79902ba77a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:28.279502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ad430-0f7f-4d32-9034-d79902ba77a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:28.279830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ad430-0f7f-4d32-9034-d79902ba77a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:28.280685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ad430-0f7f-4d32-9034-d79902ba77a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:28.283071 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ad430-0f7f-4d32-9034-d79902ba77a5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:28.296283 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 813/1602] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:28.300289 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87052dd-fa08-4bcc-ad81-49134e41a9f3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:28.312299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87052dd-fa08-4bcc-ad81-49134e41a9f3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:28.317653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47c459-489b-4f75-bbdb-49c32632edb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:28.318317 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47c459-489b-4f75-bbdb-49c32632edb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:28.318594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47c459-489b-4f75-bbdb-49c32632edb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:28.318928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47c459-489b-4f75-bbdb-49c32632edb0 admin admin] PATH_INFO: `/v3/projects/42426cccf1e5436d933f6f15321c9ed2/users/7fe4f01497904b1899f13aafbbbaf7d5/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:28.320955 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47c459-489b-4f75-bbdb-49c32632edb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42426cccf1e5436d933f6f15321c9ed2, user_id=7fe4f01497904b1899f13aafbbbaf7d5, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:28.323849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47c459-489b-4f75-bbdb-49c32632edb0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:28.349330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 790/1603] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:28 2022] PUT /identity/v3/projects/42426cccf1e5436d933f6f15321c9ed2/users/7fe4f01497904b1899f13aafbbbaf7d5/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:28.355352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181fbc-e587-46a9-ad56-8fbd9648ef77 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:28.355743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181fbc-e587-46a9-ad56-8fbd9648ef77 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:28.356132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181fbc-e587-46a9-ad56-8fbd9648ef77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:28.394217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-58181fbc-e587-46a9-ad56-8fbd9648ef77 None None] MFA Rules not processed for user `7fe4f01497904b1899f13aafbbbaf7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:28.395496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-58181fbc-e587-46a9-ad56-8fbd9648ef77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:28.468845 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 814/1604] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:28.481158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47c459-489b-4f75-bbdb-49c32632edb0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:28.482991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cf47c459-489b-4f75-bbdb-49c32632edb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:28.544525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47c459-489b-4f75-bbdb-49c32632edb0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:28.548171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931a447-de51-4993-9914-3fc8a9a46434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:28.548780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931a447-de51-4993-9914-3fc8a9a46434 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:28.549057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931a447-de51-4993-9914-3fc8a9a46434 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:28.549361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931a447-de51-4993-9914-3fc8a9a46434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:28.564845 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931a447-de51-4993-9914-3fc8a9a46434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:28.564845 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931a447-de51-4993-9914-3fc8a9a46434 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:28.572880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 791/1605] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:28.709479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181fbc-e587-46a9-ad56-8fbd9648ef77 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:28.719176 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181fbc-e587-46a9-ad56-8fbd9648ef77 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:28.723408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92119d9d-dfd2-4e56-a052-f8c3dece526d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:28.724022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92119d9d-dfd2-4e56-a052-f8c3dece526d service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:28.724306 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92119d9d-dfd2-4e56-a052-f8c3dece526d service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:28.724581 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92119d9d-dfd2-4e56-a052-f8c3dece526d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:28.785583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92119d9d-dfd2-4e56-a052-f8c3dece526d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:28.786960 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92119d9d-dfd2-4e56-a052-f8c3dece526d service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:28.803480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 815/1606] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:29.160381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931a447-de51-4993-9914-3fc8a9a46434 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:29.161868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1931a447-de51-4993-9914-3fc8a9a46434 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:29.173591 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931a447-de51-4993-9914-3fc8a9a46434 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:29.177589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d76503-cc52-499c-8f20-1b8c79581df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:29.178203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76503-cc52-499c-8f20-1b8c79581df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:29.178693 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76503-cc52-499c-8f20-1b8c79581df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:29.178971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76503-cc52-499c-8f20-1b8c79581df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:29.180765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-35d76503-cc52-499c-8f20-1b8c79581df5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:29.277940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d76503-cc52-499c-8f20-1b8c79581df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:29.279393 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d76503-cc52-499c-8f20-1b8c79581df5 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:29.302175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 792/1607] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:29.880602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92119d9d-dfd2-4e56-a052-f8c3dece526d service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:29.932002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:29.952828 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92119d9d-dfd2-4e56-a052-f8c3dece526d service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:29.965053 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:29.965935 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:29.966215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:29.966493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] PATH_INFO: `/v3/users/ba9e8dbc77fc4772a127391a70697909` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:29.967660 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba9e8dbc77fc4772a127391a70697909)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:29.970621 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:30.174278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba9e8dbc77fc4772a127391a70697909'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:30.182542 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba9e8dbc77fc4772a127391a70697909'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:30.183678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba9e8dbc77fc4772a127391a70697909'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:30.185114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba9e8dbc77fc4772a127391a70697909'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:30.272506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba9e8dbc77fc4772a127391a70697909'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:30.302566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 816/1608] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:29 2022] DELETE /identity/v3/users/ba9e8dbc77fc4772a127391a70697909 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:30.307292 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86c37-00aa-4e4f-9eba-7d3709132149 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:30.307769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86c37-00aa-4e4f-9eba-7d3709132149 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:30.307875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86c37-00aa-4e4f-9eba-7d3709132149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:30.316758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:30.337795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6df86c37-00aa-4e4f-9eba-7d3709132149 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:30.339335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6df86c37-00aa-4e4f-9eba-7d3709132149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:30.401913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 793/1609] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:30.418430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:30.419769 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:30.493648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc11a4-27f4-4e92-aa06-216e00dfa8b0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:30.498105 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212636fb-4244-4384-864f-fc4b727aab71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:30.499473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212636fb-4244-4384-864f-fc4b727aab71 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:30.500258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212636fb-4244-4384-864f-fc4b727aab71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:30.500258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212636fb-4244-4384-864f-fc4b727aab71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:30.512161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212636fb-4244-4384-864f-fc4b727aab71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:30.513672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212636fb-4244-4384-864f-fc4b727aab71 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:30.524359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 817/1610] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:31.111485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86c37-00aa-4e4f-9eba-7d3709132149 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:31.112913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6df86c37-00aa-4e4f-9eba-7d3709132149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:31.125014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86c37-00aa-4e4f-9eba-7d3709132149 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:31.130920 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:31.133332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:31.133332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:31.133332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] PATH_INFO: `/v3/projects/c6d8449ffd4e4727b7b20eda3f02c287` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:31.134110 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d8449ffd4e4727b7b20eda3f02c287)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:31.136010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:31.244914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d8449ffd4e4727b7b20eda3f02c287', 'request_id': 'req-f9a339e1-1378-4b45-a79d-0fba10f5249b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.252561 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d8449ffd4e4727b7b20eda3f02c287', 'request_id': 'req-f9a339e1-1378-4b45-a79d-0fba10f5249b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.332629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d8449ffd4e4727b7b20eda3f02c287', 'request_id': 'req-f9a339e1-1378-4b45-a79d-0fba10f5249b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.344040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] The token cache is being invalidate because project c6d8449ffd4e4727b7b20eda3f02c287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:31.344984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.345652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.347179 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 794/1611] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:31 2022] DELETE /identity/v3/projects/c6d8449ffd4e4727b7b20eda3f02c287 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:31.351222 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212636fb-4244-4384-864f-fc4b727aab71 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:31.361648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212636fb-4244-4384-864f-fc4b727aab71 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:31.429006 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:31.429701 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:31.429984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:31.430262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] PATH_INFO: `/v3/users/7fe4f01497904b1899f13aafbbbaf7d5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:31.431776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fe4f01497904b1899f13aafbbbaf7d5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:31.433730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:31.436901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:31.632956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fe4f01497904b1899f13aafbbbaf7d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.640494 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fe4f01497904b1899f13aafbbbaf7d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.640908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fe4f01497904b1899f13aafbbbaf7d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.642250 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fe4f01497904b1899f13aafbbbaf7d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.716457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fe4f01497904b1899f13aafbbbaf7d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:31.739289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 818/1612] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:31 2022] DELETE /identity/v3/users/7fe4f01497904b1899f13aafbbbaf7d5 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:32.195662 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:32.197057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:32.250880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:32.255014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:32.311502 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a339e1-1378-4b45-a79d-0fba10f5249b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:32.397152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2e495-44ac-4e42-b4bb-43640fede13c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:32.402236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:32.403300 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:32.403673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:32.403935 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] PATH_INFO: `/v3/projects/42426cccf1e5436d933f6f15321c9ed2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:32.404566 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:32.405367 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] RBAC: Authorizing `identity:delete_project(project_id=42426cccf1e5436d933f6f15321c9ed2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:32.407586 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:32.429730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312f3e8-1bd3-422b-9506-924623a182fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:32.430418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312f3e8-1bd3-422b-9506-924623a182fc service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:32.430724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312f3e8-1bd3-422b-9506-924623a182fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:32.430971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312f3e8-1bd3-422b-9506-924623a182fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:32.432772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0312f3e8-1bd3-422b-9506-924623a182fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:32.495888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312f3e8-1bd3-422b-9506-924623a182fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:32.497361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312f3e8-1bd3-422b-9506-924623a182fc service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:32.507312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 795/1613] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:32 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:32.511408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42426cccf1e5436d933f6f15321c9ed2', 'request_id': 'req-7fb78eae-89c5-44b6-8186-c3b7e1087daf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:32.595736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42426cccf1e5436d933f6f15321c9ed2', 'request_id': 'req-7fb78eae-89c5-44b6-8186-c3b7e1087daf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:32.602881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42426cccf1e5436d933f6f15321c9ed2', 'request_id': 'req-7fb78eae-89c5-44b6-8186-c3b7e1087daf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:32.615092 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] The token cache is being invalidate because project 42426cccf1e5436d933f6f15321c9ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:32.615393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:32.616087 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7fb78eae-89c5-44b6-8186-c3b7e1087daf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:32.617712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 819/1614] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:32 2022] DELETE /identity/v3/projects/42426cccf1e5436d933f6f15321c9ed2 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:32.622618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca378a-b583-4fdf-92f4-15df79c476b1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:32.622908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca378a-b583-4fdf-92f4-15df79c476b1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:32.623182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca378a-b583-4fdf-92f4-15df79c476b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:32.643737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-14ca378a-b583-4fdf-92f4-15df79c476b1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:32.644880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-14ca378a-b583-4fdf-92f4-15df79c476b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:32.710397 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 796/1615] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:32.718537 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40261-aeac-4ac7-9355-1ec660d1759d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:32.718824 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40261-aeac-4ac7-9355-1ec660d1759d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:32.719099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40261-aeac-4ac7-9355-1ec660d1759d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:32.739405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a0a40261-aeac-4ac7-9355-1ec660d1759d None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:32.740598 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a0a40261-aeac-4ac7-9355-1ec660d1759d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:32.748430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 820/1616] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:32.753403 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca378a-b583-4fdf-92f4-15df79c476b1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:32.763147 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca378a-b583-4fdf-92f4-15df79c476b1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:32.769152 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677c7f5-3c7e-4488-8807-b2208a82c532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:32.769873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677c7f5-3c7e-4488-8807-b2208a82c532 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:32.770173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677c7f5-3c7e-4488-8807-b2208a82c532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:32.770453 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677c7f5-3c7e-4488-8807-b2208a82c532 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:32.771260 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677c7f5-3c7e-4488-8807-b2208a82c532 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:32.771572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677c7f5-3c7e-4488-8807-b2208a82c532 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:32.773355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677c7f5-3c7e-4488-8807-b2208a82c532 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:32.788412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 797/1617] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:32.790431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a40261-aeac-4ac7-9355-1ec660d1759d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:32.796171 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:32.800676 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a40261-aeac-4ac7-9355-1ec660d1759d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:32.805571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525201a-817a-4fea-9689-291811563f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:32.806169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525201a-817a-4fea-9689-291811563f59 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:32.806450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525201a-817a-4fea-9689-291811563f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:32.806722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525201a-817a-4fea-9689-291811563f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:32.807362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525201a-817a-4fea-9689-291811563f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:32.809305 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525201a-817a-4fea-9689-291811563f59 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:32.887024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 821/1618] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:32 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:32.892615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677c7f5-3c7e-4488-8807-b2208a82c532 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:32.928465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677c7f5-3c7e-4488-8807-b2208a82c532 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:33.019803 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018fde7-b981-4af3-a6f0-59a3706da6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:33.020429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018fde7-b981-4af3-a6f0-59a3706da6e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:33.020711 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018fde7-b981-4af3-a6f0-59a3706da6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:33.020988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018fde7-b981-4af3-a6f0-59a3706da6e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:33.021718 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018fde7-b981-4af3-a6f0-59a3706da6e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:33.023931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018fde7-b981-4af3-a6f0-59a3706da6e4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:33.939889 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 798/1619] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:32 2022] POST /identity/v3/users => generated 293 bytes in 1048 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:33.946000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525201a-817a-4fea-9689-291811563f59 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:33.956975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525201a-817a-4fea-9689-291811563f59 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:33.961592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d3ce0-adce-4de9-a611-21cdc19b663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:33.962186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d3ce0-adce-4de9-a611-21cdc19b663b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:33.962467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d3ce0-adce-4de9-a611-21cdc19b663b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:33.962761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d3ce0-adce-4de9-a611-21cdc19b663b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:33.963381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d3ce0-adce-4de9-a611-21cdc19b663b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:33.964993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d3ce0-adce-4de9-a611-21cdc19b663b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:33.975489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 822/1620] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:33.979565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018fde7-b981-4af3-a6f0-59a3706da6e4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:33.990517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018fde7-b981-4af3-a6f0-59a3706da6e4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:33.996213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523ecd2-d762-46a1-b2c5-4058ae67efbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:33.996945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523ecd2-d762-46a1-b2c5-4058ae67efbd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:33.997115 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523ecd2-d762-46a1-b2c5-4058ae67efbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:33.997442 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523ecd2-d762-46a1-b2c5-4058ae67efbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:33.998135 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523ecd2-d762-46a1-b2c5-4058ae67efbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:33.999740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523ecd2-d762-46a1-b2c5-4058ae67efbd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:34.015679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 799/1621] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:34.017818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d3ce0-adce-4de9-a611-21cdc19b663b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:34.027981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d3ce0-adce-4de9-a611-21cdc19b663b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:34.033000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ed7f7-648e-443b-91b1-600f0091f47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:34.033747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed7f7-648e-443b-91b1-600f0091f47d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:34.041329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed7f7-648e-443b-91b1-600f0091f47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:34.041329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed7f7-648e-443b-91b1-600f0091f47d admin admin] PATH_INFO: `/v3/projects/3fe96e8d8ea44e0b9e1dcc0efbb83250/users/014f45dfa52942d0901bcc300af09c48/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:34.051131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ed7f7-648e-443b-91b1-600f0091f47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe96e8d8ea44e0b9e1dcc0efbb83250, user_id=014f45dfa52942d0901bcc300af09c48, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:34.054229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ed7f7-648e-443b-91b1-600f0091f47d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:34.158445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 823/1622] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:34 2022] PUT /identity/v3/projects/3fe96e8d8ea44e0b9e1dcc0efbb83250/users/014f45dfa52942d0901bcc300af09c48/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:34.167922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b1661-e91c-41cf-9504-cacc190442c0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:34.168209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b1661-e91c-41cf-9504-cacc190442c0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:34.168482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b1661-e91c-41cf-9504-cacc190442c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:34.177951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:34.199348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e95b1661-e91c-41cf-9504-cacc190442c0 None None] MFA Rules not processed for user `014f45dfa52942d0901bcc300af09c48`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:34.201373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e95b1661-e91c-41cf-9504-cacc190442c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:34.284508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 800/1623] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:34.288818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ed7f7-648e-443b-91b1-600f0091f47d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:34.363869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ed7f7-648e-443b-91b1-600f0091f47d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:34.369234 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cff524-705a-4dd3-8619-ed9e44b09b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:34.369677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cff524-705a-4dd3-8619-ed9e44b09b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:34.369954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cff524-705a-4dd3-8619-ed9e44b09b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:34.370229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cff524-705a-4dd3-8619-ed9e44b09b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:34.370874 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cff524-705a-4dd3-8619-ed9e44b09b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:34.372928 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cff524-705a-4dd3-8619-ed9e44b09b58 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:34.491110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 824/1624] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:34 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:34.494675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b1661-e91c-41cf-9504-cacc190442c0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:34.505737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b1661-e91c-41cf-9504-cacc190442c0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:34.573106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eac2f5-a5e2-4ee0-b9d6-6e2b891ce847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:34.573776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eac2f5-a5e2-4ee0-b9d6-6e2b891ce847 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:34.574169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eac2f5-a5e2-4ee0-b9d6-6e2b891ce847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:34.574451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eac2f5-a5e2-4ee0-b9d6-6e2b891ce847 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:34.575156 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eac2f5-a5e2-4ee0-b9d6-6e2b891ce847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:34.577100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eac2f5-a5e2-4ee0-b9d6-6e2b891ce847 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:34.668045 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 801/1625] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:34 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:34.669320 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cff524-705a-4dd3-8619-ed9e44b09b58 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:34.680738 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cff524-705a-4dd3-8619-ed9e44b09b58 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:34.685430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987755e-047f-4313-91c4-b6e03a546798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:34.686078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987755e-047f-4313-91c4-b6e03a546798 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:34.686364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987755e-047f-4313-91c4-b6e03a546798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:34.686670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987755e-047f-4313-91c4-b6e03a546798 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:34.687314 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987755e-047f-4313-91c4-b6e03a546798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:34.688969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987755e-047f-4313-91c4-b6e03a546798 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:34.700759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 825/1626] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:34.705264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eac2f5-a5e2-4ee0-b9d6-6e2b891ce847 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:34.715219 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eac2f5-a5e2-4ee0-b9d6-6e2b891ce847 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:34.720412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3d0e4-d787-4720-8d6d-e1d5b19709b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:34.724923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3d0e4-d787-4720-8d6d-e1d5b19709b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:34.724923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3d0e4-d787-4720-8d6d-e1d5b19709b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:34.724923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3d0e4-d787-4720-8d6d-e1d5b19709b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:34.724923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3d0e4-d787-4720-8d6d-e1d5b19709b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:34.724923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3d0e4-d787-4720-8d6d-e1d5b19709b8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:34.734031 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 802/1627] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:34.738257 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987755e-047f-4313-91c4-b6e03a546798 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:34.748206 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987755e-047f-4313-91c4-b6e03a546798 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:34.752748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26838806-00bf-4a52-a5b6-5d4ec934c7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:34.753436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26838806-00bf-4a52-a5b6-5d4ec934c7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:34.753742 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26838806-00bf-4a52-a5b6-5d4ec934c7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:34.754014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26838806-00bf-4a52-a5b6-5d4ec934c7f5 admin admin] PATH_INFO: `/v3/projects/e75152bddeb5411899e496586ef6907b/users/387e74f1de0643a6a47f153857496026/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:34.772312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26838806-00bf-4a52-a5b6-5d4ec934c7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e75152bddeb5411899e496586ef6907b, user_id=387e74f1de0643a6a47f153857496026, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:34.775168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26838806-00bf-4a52-a5b6-5d4ec934c7f5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:34.891968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 826/1628] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:34 2022] PUT /identity/v3/projects/e75152bddeb5411899e496586ef6907b/users/387e74f1de0643a6a47f153857496026/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:34.902061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63cee3-cdd4-45a6-9538-4e3424161d44 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:34.902677 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63cee3-cdd4-45a6-9538-4e3424161d44 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:34.903020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63cee3-cdd4-45a6-9538-4e3424161d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:34.943696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cc63cee3-cdd4-45a6-9538-4e3424161d44 None None] MFA Rules not processed for user `387e74f1de0643a6a47f153857496026`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:34.945027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cc63cee3-cdd4-45a6-9538-4e3424161d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:35.052783 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 803/1629] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:35.053496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26838806-00bf-4a52-a5b6-5d4ec934c7f5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:35.138151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:35.142072 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26838806-00bf-4a52-a5b6-5d4ec934c7f5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:35.146776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002f3ae-e51c-4380-8c8a-7b4f5a3f9f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:35.147390 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002f3ae-e51c-4380-8c8a-7b4f5a3f9f6a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:35.147670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002f3ae-e51c-4380-8c8a-7b4f5a3f9f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:35.147946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002f3ae-e51c-4380-8c8a-7b4f5a3f9f6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:35.148822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002f3ae-e51c-4380-8c8a-7b4f5a3f9f6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:35.151434 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002f3ae-e51c-4380-8c8a-7b4f5a3f9f6a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:35.244537 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 827/1630] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:35 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:35.250050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63cee3-cdd4-45a6-9538-4e3424161d44 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:35.273833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:35.275677 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63cee3-cdd4-45a6-9538-4e3424161d44 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:35.335146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d861d6d-ccf2-49c6-97a2-e26a3ce8dd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:35.335770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d861d6d-ccf2-49c6-97a2-e26a3ce8dd56 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:35.336053 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d861d6d-ccf2-49c6-97a2-e26a3ce8dd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:35.336326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d861d6d-ccf2-49c6-97a2-e26a3ce8dd56 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:35.336958 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d861d6d-ccf2-49c6-97a2-e26a3ce8dd56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:35.339024 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d861d6d-ccf2-49c6-97a2-e26a3ce8dd56 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:35.419767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 804/1631] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:35 2022] POST /identity/v3/users => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:35.425683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002f3ae-e51c-4380-8c8a-7b4f5a3f9f6a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:35.439876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002f3ae-e51c-4380-8c8a-7b4f5a3f9f6a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:35.444720 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76027c7-82ee-4ec3-b336-3c2019e03918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:35.445548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76027c7-82ee-4ec3-b336-3c2019e03918 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:35.445866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76027c7-82ee-4ec3-b336-3c2019e03918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:35.446211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76027c7-82ee-4ec3-b336-3c2019e03918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:35.447023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76027c7-82ee-4ec3-b336-3c2019e03918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:35.448589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76027c7-82ee-4ec3-b336-3c2019e03918 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:35.460304 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 828/1632] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:35.464619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d861d6d-ccf2-49c6-97a2-e26a3ce8dd56 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:35.474767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d861d6d-ccf2-49c6-97a2-e26a3ce8dd56 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:35.479398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb2d25-afeb-4601-ab07-69e1c3f35224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:35.480155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb2d25-afeb-4601-ab07-69e1c3f35224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:35.480465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb2d25-afeb-4601-ab07-69e1c3f35224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:35.480742 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb2d25-afeb-4601-ab07-69e1c3f35224 admin admin] PATH_INFO: `/v3/projects/50ee57951ead4a7b8a86bda7a7e40877/users/635927a4f044413cacf10a41cf798683/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:35.499144 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb2d25-afeb-4601-ab07-69e1c3f35224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50ee57951ead4a7b8a86bda7a7e40877, user_id=635927a4f044413cacf10a41cf798683, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:35.502931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb2d25-afeb-4601-ab07-69e1c3f35224 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:35.577341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 805/1633] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:35 2022] PUT /identity/v3/projects/50ee57951ead4a7b8a86bda7a7e40877/users/635927a4f044413cacf10a41cf798683/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:35.581651 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76027c7-82ee-4ec3-b336-3c2019e03918 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:35.593316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76027c7-82ee-4ec3-b336-3c2019e03918 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:35.660446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4191bc8-8377-4166-9996-b2cb0385af7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:35.661073 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4191bc8-8377-4166-9996-b2cb0385af7a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:35.662467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4191bc8-8377-4166-9996-b2cb0385af7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:35.662467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4191bc8-8377-4166-9996-b2cb0385af7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:35.664802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4191bc8-8377-4166-9996-b2cb0385af7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:35.667493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4191bc8-8377-4166-9996-b2cb0385af7a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:35.680283 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 829/1634] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:35.684522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb2d25-afeb-4601-ab07-69e1c3f35224 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:35.702158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb2d25-afeb-4601-ab07-69e1c3f35224 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:35.709231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9f4e2-48f4-46f9-b183-c5d2437ce94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:35.709965 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9f4e2-48f4-46f9-b183-c5d2437ce94a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:35.710907 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9f4e2-48f4-46f9-b183-c5d2437ce94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:35.711409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9f4e2-48f4-46f9-b183-c5d2437ce94a admin admin] PATH_INFO: `/v3/projects/50ee57951ead4a7b8a86bda7a7e40877/users/635927a4f044413cacf10a41cf798683/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:35.714146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9f4e2-48f4-46f9-b183-c5d2437ce94a admin admin] RBAC: Authorizing `identity:create_grant(project_id=50ee57951ead4a7b8a86bda7a7e40877, user_id=635927a4f044413cacf10a41cf798683, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:35.719689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9f4e2-48f4-46f9-b183-c5d2437ce94a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:35.729739 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 806/1635] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:35 2022] PUT /identity/v3/projects/50ee57951ead4a7b8a86bda7a7e40877/users/635927a4f044413cacf10a41cf798683/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:35.741789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835092f7-78a0-42db-895c-200be1b300c7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:35.742933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835092f7-78a0-42db-895c-200be1b300c7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:35.743467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835092f7-78a0-42db-895c-200be1b300c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:35.790512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-835092f7-78a0-42db-895c-200be1b300c7 None None] MFA Rules not processed for user `635927a4f044413cacf10a41cf798683`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:35.792565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-835092f7-78a0-42db-895c-200be1b300c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:35.894399 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 830/1636] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:35.907551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9f4e2-48f4-46f9-b183-c5d2437ce94a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:35.910095 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b7e9f4e2-48f4-46f9-b183-c5d2437ce94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:36.007794 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9f4e2-48f4-46f9-b183-c5d2437ce94a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:36.011927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952e7f8-5317-491d-a630-1e8b42fe93fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:36.026029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952e7f8-5317-491d-a630-1e8b42fe93fe service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:36.026029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952e7f8-5317-491d-a630-1e8b42fe93fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:36.026029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952e7f8-5317-491d-a630-1e8b42fe93fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:36.086333 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952e7f8-5317-491d-a630-1e8b42fe93fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:36.087790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952e7f8-5317-491d-a630-1e8b42fe93fe service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:36.100364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 807/1637] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:37.725101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835092f7-78a0-42db-895c-200be1b300c7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:37.727280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-835092f7-78a0-42db-895c-200be1b300c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:37.796413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:37.816562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835092f7-78a0-42db-895c-200be1b300c7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:37.822706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:37.824048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:37.824587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:37.825080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] PATH_INFO: `/v3/users/66fd41053c8845a3bb10e8899af2defb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:37.826463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66fd41053c8845a3bb10e8899af2defb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:37.829492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:38.058526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66fd41053c8845a3bb10e8899af2defb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.069773 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66fd41053c8845a3bb10e8899af2defb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.071587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66fd41053c8845a3bb10e8899af2defb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.073167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66fd41053c8845a3bb10e8899af2defb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.150204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66fd41053c8845a3bb10e8899af2defb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.189865 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 831/1638] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:37 2022] DELETE /identity/v3/users/66fd41053c8845a3bb10e8899af2defb => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:38.196527 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cb204-d70e-4856-b866-8dadcb729366 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:38.196997 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cb204-d70e-4856-b866-8dadcb729366 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:38.197075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cb204-d70e-4856-b866-8dadcb729366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:38.220944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-720cb204-d70e-4856-b866-8dadcb729366 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:38.223055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-720cb204-d70e-4856-b866-8dadcb729366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:38.230159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:38.231577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:38.237390 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:38.364923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d771ad-9ac9-4ccc-85d3-a150b1f99c11 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:38.373275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:38.382719 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 808/1639] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:38.396045 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cb204-d70e-4856-b866-8dadcb729366 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:38.399518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:38.399518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:38.399889 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-720cb204-d70e-4856-b866-8dadcb729366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:38.400027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] PATH_INFO: `/v3/users/ad3e6b90fa584f2189e520475d6a6ea9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:38.402188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3e6b90fa584f2189e520475d6a6ea9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:38.404454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:38.487360 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cb204-d70e-4856-b866-8dadcb729366 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:38.495918 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71cb19-d713-4e4f-a589-0e989cd975b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:38.496574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71cb19-d713-4e4f-a589-0e989cd975b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:38.496943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71cb19-d713-4e4f-a589-0e989cd975b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:38.497244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71cb19-d713-4e4f-a589-0e989cd975b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:38.511506 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71cb19-d713-4e4f-a589-0e989cd975b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:38.513347 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71cb19-d713-4e4f-a589-0e989cd975b4 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:38.529209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 809/1640] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:38.659492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3e6b90fa584f2189e520475d6a6ea9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.671333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3e6b90fa584f2189e520475d6a6ea9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.672084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3e6b90fa584f2189e520475d6a6ea9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.673568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3e6b90fa584f2189e520475d6a6ea9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.741405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3e6b90fa584f2189e520475d6a6ea9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:38.767983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:38.781324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 832/1641] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:38 2022] DELETE /identity/v3/users/ad3e6b90fa584f2189e520475d6a6ea9 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:38.786356 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2b943-ce5f-4f9c-a2e4-25f65bd65413 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:38.786942 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2b943-ce5f-4f9c-a2e4-25f65bd65413 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:38.787239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2b943-ce5f-4f9c-a2e4-25f65bd65413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:38.838549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3ae2b943-ce5f-4f9c-a2e4-25f65bd65413 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:38.839801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3ae2b943-ce5f-4f9c-a2e4-25f65bd65413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:38.994113 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 810/1642] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:39.003393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:39.005832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:39.076457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2b943-ce5f-4f9c-a2e4-25f65bd65413 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:39.077910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3ae2b943-ce5f-4f9c-a2e4-25f65bd65413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:39.097926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2b943-ce5f-4f9c-a2e4-25f65bd65413 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:39.119627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:39.120293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:39.120569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:39.120841 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] PATH_INFO: `/v3/projects/77c355899b8540d99ba3638c329d8def` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:39.125306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77c355899b8540d99ba3638c329d8def)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:39.129726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:39.150412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e06353-79e0-4683-b76f-e0e818108b61 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:39.161065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910abae-3d97-41a7-8b98-ac572af47aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:39.161730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910abae-3d97-41a7-8b98-ac572af47aac service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:39.162020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910abae-3d97-41a7-8b98-ac572af47aac service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:39.162274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910abae-3d97-41a7-8b98-ac572af47aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:39.189358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910abae-3d97-41a7-8b98-ac572af47aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:39.189358 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910abae-3d97-41a7-8b98-ac572af47aac service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:39.203401 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 833/1643] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:39.324518 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:39.333351 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77c355899b8540d99ba3638c329d8def', 'request_id': 'req-d53d09f3-fe01-4737-a33c-5dc44a3cff66'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.340328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77c355899b8540d99ba3638c329d8def', 'request_id': 'req-d53d09f3-fe01-4737-a33c-5dc44a3cff66'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.418006 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910abae-3d97-41a7-8b98-ac572af47aac service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:39.419066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2910abae-3d97-41a7-8b98-ac572af47aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:39.463870 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77c355899b8540d99ba3638c329d8def', 'request_id': 'req-d53d09f3-fe01-4737-a33c-5dc44a3cff66'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.483812 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] The token cache is being invalidate because project 77c355899b8540d99ba3638c329d8def was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:39.484151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.487631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.490868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 811/1644] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:39 2022] DELETE /identity/v3/projects/77c355899b8540d99ba3638c329d8def => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:39.498724 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:39.503563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:39.542037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910abae-3d97-41a7-8b98-ac572af47aac service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:39.602020 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d09f3-fe01-4737-a33c-5dc44a3cff66 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:39.608253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:39.612063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:39.612063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:39.612063 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] PATH_INFO: `/v3/users/5baff42ecf8741f5bcb72fdfaf4b3ca4` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:39.612696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5baff42ecf8741f5bcb72fdfaf4b3ca4)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:39.615114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:39.663954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:39.664714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:39.665096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:39.665507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:39.668263 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:39.761682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:39.763379 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:39.780012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 834/1645] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:39.845779 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5baff42ecf8741f5bcb72fdfaf4b3ca4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.846355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5baff42ecf8741f5bcb72fdfaf4b3ca4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.858866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5baff42ecf8741f5bcb72fdfaf4b3ca4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.861703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5baff42ecf8741f5bcb72fdfaf4b3ca4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.933006 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5baff42ecf8741f5bcb72fdfaf4b3ca4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:39.964592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 812/1646] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:39 2022] DELETE /identity/v3/users/5baff42ecf8741f5bcb72fdfaf4b3ca4 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:40.387109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:40.389106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:40.402927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:40.408689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be0074-d7da-4428-b588-13e79557d151 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:40.497766 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:40.499463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:40.499463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:40.499463 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] PATH_INFO: `/v3/projects/a9a6240e83f14215a04fecee2306f7f1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:40.501646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a6240e83f14215a04fecee2306f7f1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:40.504241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:40.638766 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a6240e83f14215a04fecee2306f7f1', 'request_id': 'req-610894c8-dc30-4d58-8b00-57bd9580e1b0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:40.685562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:40.687145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:40.712582 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3c6f1-c737-44d1-945c-4c6e583d6380 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:40.727865 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a6240e83f14215a04fecee2306f7f1', 'request_id': 'req-610894c8-dc30-4d58-8b00-57bd9580e1b0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:40.736565 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a6240e83f14215a04fecee2306f7f1', 'request_id': 'req-610894c8-dc30-4d58-8b00-57bd9580e1b0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:40.759166 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] The token cache is being invalidate because project a9a6240e83f14215a04fecee2306f7f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:40.764435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:40.764435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:40.765559 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 835/1647] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:40 2022] DELETE /identity/v3/projects/a9a6240e83f14215a04fecee2306f7f1 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:40.768621 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:40.772149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:40.836545 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:40.847023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:40.848594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:40.849433 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:40.849886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] PATH_INFO: `/v3/projects/2ee56d1bead14af5b914648b55b672ba` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:40.852189 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee56d1bead14af5b914648b55b672ba)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:40.857888 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:40.912163 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610894c8-dc30-4d58-8b00-57bd9580e1b0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:40.918552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:40.919357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:40.919734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:40.920677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] PATH_INFO: `/v3/users/a31d6c721707462ebfeee8c40dea7d01` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:40.922099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31d6c721707462ebfeee8c40dea7d01)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:40.930820 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:40.995173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee56d1bead14af5b914648b55b672ba', 'request_id': 'req-5c7ae923-b489-4d16-a23e-8e87c6f997c0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.007668 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee56d1bead14af5b914648b55b672ba', 'request_id': 'req-5c7ae923-b489-4d16-a23e-8e87c6f997c0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.133610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee56d1bead14af5b914648b55b672ba', 'request_id': 'req-5c7ae923-b489-4d16-a23e-8e87c6f997c0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.162042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] The token cache is being invalidate because project 2ee56d1bead14af5b914648b55b672ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:41.164923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.172837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5c7ae923-b489-4d16-a23e-8e87c6f997c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.172837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 813/1648] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:40 2022] DELETE /identity/v3/projects/2ee56d1bead14af5b914648b55b672ba => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:41.177903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525af304-3d70-4083-bf84-c9648d001b6c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:41.178372 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525af304-3d70-4083-bf84-c9648d001b6c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:41.179473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525af304-3d70-4083-bf84-c9648d001b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:41.210242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-525af304-3d70-4083-bf84-c9648d001b6c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:41.211544 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-525af304-3d70-4083-bf84-c9648d001b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:41.225804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:41.246111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31d6c721707462ebfeee8c40dea7d01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.273521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31d6c721707462ebfeee8c40dea7d01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.274610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31d6c721707462ebfeee8c40dea7d01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.275453 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31d6c721707462ebfeee8c40dea7d01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.298435 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 814/1649] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:41.308138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432619d7-d86d-4c92-ba3c-f92f5560b625 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:41.311434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432619d7-d86d-4c92-ba3c-f92f5560b625 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:41.311434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432619d7-d86d-4c92-ba3c-f92f5560b625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:41.341059 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-432619d7-d86d-4c92-ba3c-f92f5560b625 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:41.342772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-432619d7-d86d-4c92-ba3c-f92f5560b625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:41.360115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31d6c721707462ebfeee8c40dea7d01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:41.370978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 815/1650] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:41.372417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432619d7-d86d-4c92-ba3c-f92f5560b625 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:41.400106 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 836/1651] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:40 2022] DELETE /identity/v3/users/a31d6c721707462ebfeee8c40dea7d01 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:41.403375 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432619d7-d86d-4c92-ba3c-f92f5560b625 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:41.567601 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11090c65-02ba-411d-a911-cdf5761d25c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:41.568384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11090c65-02ba-411d-a911-cdf5761d25c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:41.568882 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11090c65-02ba-411d-a911-cdf5761d25c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:41.569253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11090c65-02ba-411d-a911-cdf5761d25c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:41.570128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11090c65-02ba-411d-a911-cdf5761d25c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:41.570508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11090c65-02ba-411d-a911-cdf5761d25c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:41.574786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11090c65-02ba-411d-a911-cdf5761d25c1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:41.595436 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:41.597555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:41.603058 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 816/1652] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:41.604885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11090c65-02ba-411d-a911-cdf5761d25c1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:41.624832 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11090c65-02ba-411d-a911-cdf5761d25c1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:41.630730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea36ec-fe11-4b6b-83a9-720797fb3e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:41.631618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea36ec-fe11-4b6b-83a9-720797fb3e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:41.631788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea36ec-fe11-4b6b-83a9-720797fb3e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:41.632111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea36ec-fe11-4b6b-83a9-720797fb3e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:41.632981 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea36ec-fe11-4b6b-83a9-720797fb3e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:41.635205 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea36ec-fe11-4b6b-83a9-720797fb3e52 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:41.664786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5629dbd-e4a1-40ec-b58a-647347c403ba admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:41.668392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864e12-2f0d-4d8b-8fc4-0d86d2cd7f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:41.668997 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864e12-2f0d-4d8b-8fc4-0d86d2cd7f3b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:41.669309 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864e12-2f0d-4d8b-8fc4-0d86d2cd7f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:41.669583 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864e12-2f0d-4d8b-8fc4-0d86d2cd7f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:41.671285 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8c864e12-2f0d-4d8b-8fc4-0d86d2cd7f3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:41.745651 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 817/1653] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:41 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:41.747887 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea36ec-fe11-4b6b-83a9-720797fb3e52 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:41.763534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c864e12-2f0d-4d8b-8fc4-0d86d2cd7f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:41.765172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c864e12-2f0d-4d8b-8fc4-0d86d2cd7f3b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:41.775986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 837/1654] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:41.848484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea36ec-fe11-4b6b-83a9-720797fb3e52 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:41.854337 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595e168-624b-482c-94ed-a08c65146c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:41.854968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595e168-624b-482c-94ed-a08c65146c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:41.855252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595e168-624b-482c-94ed-a08c65146c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:41.855528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595e168-624b-482c-94ed-a08c65146c49 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:41.856235 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595e168-624b-482c-94ed-a08c65146c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:41.858478 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595e168-624b-482c-94ed-a08c65146c49 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:41.991589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 818/1655] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:41 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:41.993052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864e12-2f0d-4d8b-8fc4-0d86d2cd7f3b service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.005389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864e12-2f0d-4d8b-8fc4-0d86d2cd7f3b service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.017158 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98988af-2d1c-4883-a98e-1d9a8f406e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.017931 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98988af-2d1c-4883-a98e-1d9a8f406e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.018118 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98988af-2d1c-4883-a98e-1d9a8f406e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.018344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98988af-2d1c-4883-a98e-1d9a8f406e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.019002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98988af-2d1c-4883-a98e-1d9a8f406e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:42.020647 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98988af-2d1c-4883-a98e-1d9a8f406e42 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:42.041385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595e168-624b-482c-94ed-a08c65146c49 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.043732 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b595e168-624b-482c-94ed-a08c65146c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:42.050187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 838/1656] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:42.055139 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98988af-2d1c-4883-a98e-1d9a8f406e42 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.068801 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98988af-2d1c-4883-a98e-1d9a8f406e42 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.072446 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595e168-624b-482c-94ed-a08c65146c49 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.077551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3c839-4932-4748-9909-88d26f256994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.078634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3c839-4932-4748-9909-88d26f256994 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.079554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3c839-4932-4748-9909-88d26f256994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.080039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3c839-4932-4748-9909-88d26f256994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.081112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3c839-4932-4748-9909-88d26f256994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:42.083780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3c839-4932-4748-9909-88d26f256994 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:42.086472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.087115 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.087464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.087652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] PATH_INFO: `/v3/projects/e4868f3cfe85453c8bfafde9cd8aba8d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.094694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4868f3cfe85453c8bfafde9cd8aba8d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:42.096856 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:42.102467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 839/1657] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:42.106740 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3c839-4932-4748-9909-88d26f256994 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.119571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3c839-4932-4748-9909-88d26f256994 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.126774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af7cdd-4bc9-4d0b-a94b-790cdc591c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.128168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af7cdd-4bc9-4d0b-a94b-790cdc591c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.128662 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af7cdd-4bc9-4d0b-a94b-790cdc591c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.129520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af7cdd-4bc9-4d0b-a94b-790cdc591c45 admin admin] PATH_INFO: `/v3/projects/9e47d86b67b24e0fa8b59d7b2822c0ae/users/e8a6ac08778e49f79a78c7faf537c3c9/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.158291 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af7cdd-4bc9-4d0b-a94b-790cdc591c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e47d86b67b24e0fa8b59d7b2822c0ae, user_id=e8a6ac08778e49f79a78c7faf537c3c9, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:42.161101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af7cdd-4bc9-4d0b-a94b-790cdc591c45 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:42.280506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 840/1658] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:42 2022] PUT /identity/v3/projects/9e47d86b67b24e0fa8b59d7b2822c0ae/users/e8a6ac08778e49f79a78c7faf537c3c9/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:42.289703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4868f3cfe85453c8bfafde9cd8aba8d', 'request_id': 'req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.294029 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cb0ad-cf4b-41aa-8cc4-7fae047923c5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.294646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cb0ad-cf4b-41aa-8cc4-7fae047923c5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.295133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cb0ad-cf4b-41aa-8cc4-7fae047923c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.297991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4868f3cfe85453c8bfafde9cd8aba8d', 'request_id': 'req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.330188 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-196cb0ad-cf4b-41aa-8cc4-7fae047923c5 None None] MFA Rules not processed for user `e8a6ac08778e49f79a78c7faf537c3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:42.331927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-196cb0ad-cf4b-41aa-8cc4-7fae047923c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:42.384895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4868f3cfe85453c8bfafde9cd8aba8d', 'request_id': 'req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.399751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] The token cache is being invalidate because project e4868f3cfe85453c8bfafde9cd8aba8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:42.400441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.401511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.403911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 819/1659] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:42 2022] DELETE /identity/v3/projects/e4868f3cfe85453c8bfafde9cd8aba8d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:42.408590 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.410646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:42.420732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 841/1660] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:42.427344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cb0ad-cf4b-41aa-8cc4-7fae047923c5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.434518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-196cb0ad-cf4b-41aa-8cc4-7fae047923c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:42.514780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b855b-0aa3-4478-9f49-bc1a1e0a8e58 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.521569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.522282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.526385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.526385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] PATH_INFO: `/v3/users/56b7f292b5ca492abc7c4b627eed88ad` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.526385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b7f292b5ca492abc7c4b627eed88ad)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:42.527273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:42.536933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cb0ad-cf4b-41aa-8cc4-7fae047923c5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.544145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4190e94-1eb3-4b39-94fa-65ce27ac23d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.544900 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4190e94-1eb3-4b39-94fa-65ce27ac23d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.545047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4190e94-1eb3-4b39-94fa-65ce27ac23d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.545373 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4190e94-1eb3-4b39-94fa-65ce27ac23d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.546020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4190e94-1eb3-4b39-94fa-65ce27ac23d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:42.548507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4190e94-1eb3-4b39-94fa-65ce27ac23d2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:42.646507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:42.654017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 842/1661] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:42 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:42.655128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4190e94-1eb3-4b39-94fa-65ce27ac23d2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.732531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4190e94-1eb3-4b39-94fa-65ce27ac23d2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.742114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637dca8-ba03-49fb-853c-2a5e107cf82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.742834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637dca8-ba03-49fb-853c-2a5e107cf82d admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.743038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637dca8-ba03-49fb-853c-2a5e107cf82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.743366 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637dca8-ba03-49fb-853c-2a5e107cf82d admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.744018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637dca8-ba03-49fb-853c-2a5e107cf82d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:42.744877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b7f292b5ca492abc7c4b627eed88ad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.745621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b7f292b5ca492abc7c4b627eed88ad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.746097 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637dca8-ba03-49fb-853c-2a5e107cf82d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:42.754037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b7f292b5ca492abc7c4b627eed88ad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.754919 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b7f292b5ca492abc7c4b627eed88ad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.824066 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b7f292b5ca492abc7c4b627eed88ad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:42.832455 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 843/1662] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:42 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:42.836288 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637dca8-ba03-49fb-853c-2a5e107cf82d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.838269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e637dca8-ba03-49fb-853c-2a5e107cf82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:42.844328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 820/1663] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:42 2022] DELETE /identity/v3/users/56b7f292b5ca492abc7c4b627eed88ad => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:42.852383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637dca8-ba03-49fb-853c-2a5e107cf82d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.910278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:42.928612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158808e-8ba7-455c-b8e9-49932beaf7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.929409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158808e-8ba7-455c-b8e9-49932beaf7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.929761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158808e-8ba7-455c-b8e9-49932beaf7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.930100 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158808e-8ba7-455c-b8e9-49932beaf7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.930890 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158808e-8ba7-455c-b8e9-49932beaf7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:42.933132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158808e-8ba7-455c-b8e9-49932beaf7e2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:42.945848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 844/1664] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:42.949672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.960363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550710b-b99e-4b4a-b69c-006da713d98c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.964927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47aa4d-c383-4d71-98ff-7a6f384fe57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.965561 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47aa4d-c383-4d71-98ff-7a6f384fe57a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.965840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47aa4d-c383-4d71-98ff-7a6f384fe57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:42.966112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47aa4d-c383-4d71-98ff-7a6f384fe57a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:42.966814 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47aa4d-c383-4d71-98ff-7a6f384fe57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:42.968438 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47aa4d-c383-4d71-98ff-7a6f384fe57a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:42.979305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 821/1665] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:42.983743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158808e-8ba7-455c-b8e9-49932beaf7e2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:42.994316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158808e-8ba7-455c-b8e9-49932beaf7e2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:42.998907 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480960b-afa6-4620-af93-36631e3f1c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:42.999560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480960b-afa6-4620-af93-36631e3f1c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:42.999840 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480960b-afa6-4620-af93-36631e3f1c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.000114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480960b-afa6-4620-af93-36631e3f1c13 admin admin] PATH_INFO: `/v3/projects/78d8f88e764e4867a7553741bf622ff3/users/d62988bfa6454a97a09b52a314292145/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.016781 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480960b-afa6-4620-af93-36631e3f1c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d8f88e764e4867a7553741bf622ff3, user_id=d62988bfa6454a97a09b52a314292145, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.019600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480960b-afa6-4620-af93-36631e3f1c13 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.151326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 845/1666] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:42 2022] PUT /identity/v3/projects/78d8f88e764e4867a7553741bf622ff3/users/d62988bfa6454a97a09b52a314292145/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:43.161729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246f2d-bcc2-47fa-8739-2435495a5343 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.162353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246f2d-bcc2-47fa-8739-2435495a5343 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.162636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246f2d-bcc2-47fa-8739-2435495a5343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.198004 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b6246f2d-bcc2-47fa-8739-2435495a5343 None None] MFA Rules not processed for user `d62988bfa6454a97a09b52a314292145`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:43.199571 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b6246f2d-bcc2-47fa-8739-2435495a5343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:43.274725 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 822/1667] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:43.279198 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480960b-afa6-4620-af93-36631e3f1c13 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.341099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480960b-afa6-4620-af93-36631e3f1c13 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.346372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15c72e-cab6-4d1f-b186-983b9aba8f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.346972 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15c72e-cab6-4d1f-b186-983b9aba8f9a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.347249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15c72e-cab6-4d1f-b186-983b9aba8f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.347520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15c72e-cab6-4d1f-b186-983b9aba8f9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.348149 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15c72e-cab6-4d1f-b186-983b9aba8f9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.350230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15c72e-cab6-4d1f-b186-983b9aba8f9a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.357688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246f2d-bcc2-47fa-8739-2435495a5343 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.359157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b6246f2d-bcc2-47fa-8739-2435495a5343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:43.381137 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246f2d-bcc2-47fa-8739-2435495a5343 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.386464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.387114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.387417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.387696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] PATH_INFO: `/v3/projects/967c2861dac34a528b80e9db018ace37` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.389047 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=967c2861dac34a528b80e9db018ace37)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.391037 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.481154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 846/1668] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:43 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:43.482273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15c72e-cab6-4d1f-b186-983b9aba8f9a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.492164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967c2861dac34a528b80e9db018ace37', 'request_id': 'req-365b8abd-fc7a-420b-a159-a830477865bd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:43.498610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967c2861dac34a528b80e9db018ace37', 'request_id': 'req-365b8abd-fc7a-420b-a159-a830477865bd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:43.521353 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:43.546208 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15c72e-cab6-4d1f-b186-983b9aba8f9a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.551816 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae307-cbce-45cc-8e4c-139ad9cb3a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.552727 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae307-cbce-45cc-8e4c-139ad9cb3a80 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.555015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae307-cbce-45cc-8e4c-139ad9cb3a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.555015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae307-cbce-45cc-8e4c-139ad9cb3a80 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.555015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae307-cbce-45cc-8e4c-139ad9cb3a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.555965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae307-cbce-45cc-8e4c-139ad9cb3a80 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.560718 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967c2861dac34a528b80e9db018ace37', 'request_id': 'req-365b8abd-fc7a-420b-a159-a830477865bd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:43.574594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] The token cache is being invalidate because project 967c2861dac34a528b80e9db018ace37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:43.574896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:43.575599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8abd-fc7a-420b-a159-a830477865bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:43.577232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 823/1669] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:43 2022] DELETE /identity/v3/projects/967c2861dac34a528b80e9db018ace37 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:43.582970 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605d8e1-8f8e-4c0b-a27a-c393b93015ed None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.583327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605d8e1-8f8e-4c0b-a27a-c393b93015ed None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.583615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605d8e1-8f8e-4c0b-a27a-c393b93015ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.603725 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a605d8e1-8f8e-4c0b-a27a-c393b93015ed None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:43.604867 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a605d8e1-8f8e-4c0b-a27a-c393b93015ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:43.613086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:43.613473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 824/1670] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:43.621620 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff576d1-d1ac-401f-b82b-d8777f312c17 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.621903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff576d1-d1ac-401f-b82b-d8777f312c17 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.623729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff576d1-d1ac-401f-b82b-d8777f312c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.641781 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fff576d1-d1ac-401f-b82b-d8777f312c17 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:43.642977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fff576d1-d1ac-401f-b82b-d8777f312c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:43.649080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 847/1671] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:43 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:43.651341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 825/1672] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:43.657685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae307-cbce-45cc-8e4c-139ad9cb3a80 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.659358 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff576d1-d1ac-401f-b82b-d8777f312c17 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.659473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1b2ae307-cbce-45cc-8e4c-139ad9cb3a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:43.668875 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff576d1-d1ac-401f-b82b-d8777f312c17 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.671512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae307-cbce-45cc-8e4c-139ad9cb3a80 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.675033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343594b-db3c-4762-9dea-978d92e1dd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.675636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343594b-db3c-4762-9dea-978d92e1dd5d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.675914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343594b-db3c-4762-9dea-978d92e1dd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.676187 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343594b-db3c-4762-9dea-978d92e1dd5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.677238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343594b-db3c-4762-9dea-978d92e1dd5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:43.677546 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343594b-db3c-4762-9dea-978d92e1dd5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.679253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343594b-db3c-4762-9dea-978d92e1dd5d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.683412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccc9f6-3036-4e10-b47c-f1b3a68fa0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.683412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccc9f6-3036-4e10-b47c-f1b3a68fa0ec admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.683412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccc9f6-3036-4e10-b47c-f1b3a68fa0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.683412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccc9f6-3036-4e10-b47c-f1b3a68fa0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.684287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccc9f6-3036-4e10-b47c-f1b3a68fa0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.684287 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccc9f6-3036-4e10-b47c-f1b3a68fa0ec admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.692448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 826/1673] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:43.697463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343594b-db3c-4762-9dea-978d92e1dd5d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.703048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 848/1674] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:43.704818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccc9f6-3036-4e10-b47c-f1b3a68fa0ec admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.710426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343594b-db3c-4762-9dea-978d92e1dd5d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.716009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccc9f6-3036-4e10-b47c-f1b3a68fa0ec admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.716488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7622-c44f-4ebd-b395-578781dcdc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.717217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7622-c44f-4ebd-b395-578781dcdc3a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.717569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7622-c44f-4ebd-b395-578781dcdc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.717995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7622-c44f-4ebd-b395-578781dcdc3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.718692 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7622-c44f-4ebd-b395-578781dcdc3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.721326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7622-c44f-4ebd-b395-578781dcdc3a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.721675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3281-c7d0-4a1d-b193-45f993ba57af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.722339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3281-c7d0-4a1d-b193-45f993ba57af admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.722669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3281-c7d0-4a1d-b193-45f993ba57af admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.722950 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3281-c7d0-4a1d-b193-45f993ba57af admin admin] PATH_INFO: `/v3/projects/e06ed4a0c9ad40cfa80e9338d299154d/users/d4b72784459d41e1b6079cc361711eca/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.740738 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e3281-c7d0-4a1d-b193-45f993ba57af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06ed4a0c9ad40cfa80e9338d299154d, user_id=d4b72784459d41e1b6079cc361711eca, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.746225 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e3281-c7d0-4a1d-b193-45f993ba57af admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.837968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 827/1675] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:43 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:43.842043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7622-c44f-4ebd-b395-578781dcdc3a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.867493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 849/1676] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:43 2022] PUT /identity/v3/projects/e06ed4a0c9ad40cfa80e9338d299154d/users/d4b72784459d41e1b6079cc361711eca/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:43.871431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3281-c7d0-4a1d-b193-45f993ba57af admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.920628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7622-c44f-4ebd-b395-578781dcdc3a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.945086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3281-c7d0-4a1d-b193-45f993ba57af admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.950715 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e07e-2df1-49c5-a315-c09694d5669c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.951511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e07e-2df1-49c5-a315-c09694d5669c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.951798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e07e-2df1-49c5-a315-c09694d5669c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.952564 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e07e-2df1-49c5-a315-c09694d5669c admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.954083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613e07e-2df1-49c5-a315-c09694d5669c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.956913 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613e07e-2df1-49c5-a315-c09694d5669c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.966571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 850/1677] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:43.971442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e07e-2df1-49c5-a315-c09694d5669c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:43.980385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98486c-8b95-4d37-8b38-09f11db06338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.981010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98486c-8b95-4d37-8b38-09f11db06338 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.981322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98486c-8b95-4d37-8b38-09f11db06338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.981600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98486c-8b95-4d37-8b38-09f11db06338 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.981690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e07e-2df1-49c5-a315-c09694d5669c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:43.982242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98486c-8b95-4d37-8b38-09f11db06338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.984201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98486c-8b95-4d37-8b38-09f11db06338 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:43.986400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a8cc8-311a-4f22-9eae-1a4c58c2a8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:43.987078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a8cc8-311a-4f22-9eae-1a4c58c2a8a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:43.987357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a8cc8-311a-4f22-9eae-1a4c58c2a8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:43.987632 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a8cc8-311a-4f22-9eae-1a4c58c2a8a9 admin admin] PATH_INFO: `/v3/projects/e06ed4a0c9ad40cfa80e9338d299154d/users/d4b72784459d41e1b6079cc361711eca/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:43.989545 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a8cc8-311a-4f22-9eae-1a4c58c2a8a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06ed4a0c9ad40cfa80e9338d299154d, user_id=d4b72784459d41e1b6079cc361711eca, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:43.992171 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a8cc8-311a-4f22-9eae-1a4c58c2a8a9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:44.002860 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 851/1678] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:43 2022] PUT /identity/v3/projects/e06ed4a0c9ad40cfa80e9338d299154d/users/d4b72784459d41e1b6079cc361711eca/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:44.014446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933035e0-a82c-4863-a690-a6a0cff30bbb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.014734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933035e0-a82c-4863-a690-a6a0cff30bbb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.015008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933035e0-a82c-4863-a690-a6a0cff30bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.044506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-933035e0-a82c-4863-a690-a6a0cff30bbb None None] MFA Rules not processed for user `d4b72784459d41e1b6079cc361711eca`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:44.045950 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-933035e0-a82c-4863-a690-a6a0cff30bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:44.048940 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:44.055463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 828/1679] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:43 2022] POST /identity/v3/users => generated 295 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:44.059606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98486c-8b95-4d37-8b38-09f11db06338 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:44.069725 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98486c-8b95-4d37-8b38-09f11db06338 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:44.117780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:44.121133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 852/1680] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:44.133557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b49d1-3964-474a-b1ee-b41ea646355e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:44.133995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933035e0-a82c-4863-a690-a6a0cff30bbb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:44.134185 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b49d1-3964-474a-b1ee-b41ea646355e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.134495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b49d1-3964-474a-b1ee-b41ea646355e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.134736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b49d1-3964-474a-b1ee-b41ea646355e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.135431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-933035e0-a82c-4863-a690-a6a0cff30bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:44.135537 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b49d1-3964-474a-b1ee-b41ea646355e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:44.137033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b49d1-3964-474a-b1ee-b41ea646355e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:44.149016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 829/1681] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:44.152959 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b49d1-3964-474a-b1ee-b41ea646355e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:44.164199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b49d1-3964-474a-b1ee-b41ea646355e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:44.169723 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defb6e0-1827-4fcd-8423-df9b508f1a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:44.170563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defb6e0-1827-4fcd-8423-df9b508f1a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.170850 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defb6e0-1827-4fcd-8423-df9b508f1a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.171175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defb6e0-1827-4fcd-8423-df9b508f1a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.171816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defb6e0-1827-4fcd-8423-df9b508f1a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:44.173570 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defb6e0-1827-4fcd-8423-df9b508f1a14 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:44.185142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 830/1682] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:44.190041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defb6e0-1827-4fcd-8423-df9b508f1a14 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:44.197957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933035e0-a82c-4863-a690-a6a0cff30bbb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:44.200182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defb6e0-1827-4fcd-8423-df9b508f1a14 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:44.201629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577797b-825f-4d58-bd57-a15f42a7a2c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:44.202237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577797b-825f-4d58-bd57-a15f42a7a2c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.202518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577797b-825f-4d58-bd57-a15f42a7a2c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.202794 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577797b-825f-4d58-bd57-a15f42a7a2c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.204648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d577797b-825f-4d58-bd57-a15f42a7a2c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:44.205577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed73a0-5f41-43f1-b010-c193160467d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:44.206233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed73a0-5f41-43f1-b010-c193160467d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.206519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed73a0-5f41-43f1-b010-c193160467d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.206801 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed73a0-5f41-43f1-b010-c193160467d2 admin admin] PATH_INFO: `/v3/projects/386dd306d30b4b61b3335c22d857a87f/users/ab0ca7470d7a47d7a8eb72c1d5bedf2a/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.222522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed73a0-5f41-43f1-b010-c193160467d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=386dd306d30b4b61b3335c22d857a87f, user_id=ab0ca7470d7a47d7a8eb72c1d5bedf2a, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:44.225214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed73a0-5f41-43f1-b010-c193160467d2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:44.263580 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577797b-825f-4d58-bd57-a15f42a7a2c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:44.265031 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577797b-825f-4d58-bd57-a15f42a7a2c9 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:44.273841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 853/1683] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:44.305081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 831/1684] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:44 2022] PUT /identity/v3/projects/386dd306d30b4b61b3335c22d857a87f/users/ab0ca7470d7a47d7a8eb72c1d5bedf2a/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:44.310371 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:44.316558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3c74-82ec-4b28-8a96-d0a8d2e1b52e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.316831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3c74-82ec-4b28-8a96-d0a8d2e1b52e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.317108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3c74-82ec-4b28-8a96-d0a8d2e1b52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.346701 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d41b3c74-82ec-4b28-8a96-d0a8d2e1b52e None None] MFA Rules not processed for user `ab0ca7470d7a47d7a8eb72c1d5bedf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:44.347916 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d41b3c74-82ec-4b28-8a96-d0a8d2e1b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:44.390419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:44.418114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 854/1685] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:44.422975 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed73a0-5f41-43f1-b010-c193160467d2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:44.485607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed73a0-5f41-43f1-b010-c193160467d2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:44.490362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf3a21-68f1-473a-bdfd-f08fc8a4ed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:44.490969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf3a21-68f1-473a-bdfd-f08fc8a4ed81 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.491370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf3a21-68f1-473a-bdfd-f08fc8a4ed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.491587 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf3a21-68f1-473a-bdfd-f08fc8a4ed81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.492235 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf3a21-68f1-473a-bdfd-f08fc8a4ed81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:44.494195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf3a21-68f1-473a-bdfd-f08fc8a4ed81 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:44.586320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 832/1686] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:44 2022] POST /identity/v3/projects => generated 389 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:44.590556 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b3c74-82ec-4b28-8a96-d0a8d2e1b52e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:44.601754 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b3c74-82ec-4b28-8a96-d0a8d2e1b52e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:44.659338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4a9b9-0f18-415a-a20d-8ff3e9811762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:44.659880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4a9b9-0f18-415a-a20d-8ff3e9811762 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.660168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4a9b9-0f18-415a-a20d-8ff3e9811762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.660441 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4a9b9-0f18-415a-a20d-8ff3e9811762 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.661068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4a9b9-0f18-415a-a20d-8ff3e9811762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:44.663060 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4a9b9-0f18-415a-a20d-8ff3e9811762 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:44.761322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 855/1687] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:44 2022] POST /identity/v3/users => generated 301 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:44.765434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf3a21-68f1-473a-bdfd-f08fc8a4ed81 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:44.775943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf3a21-68f1-473a-bdfd-f08fc8a4ed81 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:44.780583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1776eae-963e-448c-a56d-7c38f87817d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:44.781214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1776eae-963e-448c-a56d-7c38f87817d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.781530 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1776eae-963e-448c-a56d-7c38f87817d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.781902 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1776eae-963e-448c-a56d-7c38f87817d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.782739 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1776eae-963e-448c-a56d-7c38f87817d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:44.784376 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1776eae-963e-448c-a56d-7c38f87817d3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:44.794911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 833/1688] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:44.799775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4a9b9-0f18-415a-a20d-8ff3e9811762 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:44.810420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4a9b9-0f18-415a-a20d-8ff3e9811762 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:44.817558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cb5bc-58de-41a9-b473-3e0552cc28ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:44.829336 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cb5bc-58de-41a9-b473-3e0552cc28ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:44.829336 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cb5bc-58de-41a9-b473-3e0552cc28ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:44.829336 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cb5bc-58de-41a9-b473-3e0552cc28ed admin admin] PATH_INFO: `/v3/projects/89196dd924314a9d86d11e7fc1064058/users/f183a0cf63a247ceb13603576e1edb17/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:44.833852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:44.837018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cb5bc-58de-41a9-b473-3e0552cc28ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=89196dd924314a9d86d11e7fc1064058, user_id=f183a0cf63a247ceb13603576e1edb17, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:44.840065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cb5bc-58de-41a9-b473-3e0552cc28ed admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:44.928986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 856/1689] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:44 2022] PUT /identity/v3/projects/89196dd924314a9d86d11e7fc1064058/users/f183a0cf63a247ceb13603576e1edb17/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:44.932929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1776eae-963e-448c-a56d-7c38f87817d3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:44.946054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1776eae-963e-448c-a56d-7c38f87817d3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:45.012670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cceb4-4c20-4128-aca8-c2c40a1e0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:45.013422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cceb4-4c20-4128-aca8-c2c40a1e0d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:45.013704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cceb4-4c20-4128-aca8-c2c40a1e0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:45.013981 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cceb4-4c20-4128-aca8-c2c40a1e0d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:45.014629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cceb4-4c20-4128-aca8-c2c40a1e0d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:45.016289 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cceb4-4c20-4128-aca8-c2c40a1e0d89 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:45.026627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 834/1690] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:45.030591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cb5bc-58de-41a9-b473-3e0552cc28ed admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:45.042263 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cb5bc-58de-41a9-b473-3e0552cc28ed admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:45.051582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1420847-87ec-4948-b280-ffc984f65618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:45.051582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1420847-87ec-4948-b280-ffc984f65618 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:45.051582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1420847-87ec-4948-b280-ffc984f65618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:45.051582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1420847-87ec-4948-b280-ffc984f65618 admin admin] PATH_INFO: `/v3/projects/89196dd924314a9d86d11e7fc1064058/users/f183a0cf63a247ceb13603576e1edb17/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:45.052211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1420847-87ec-4948-b280-ffc984f65618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89196dd924314a9d86d11e7fc1064058, user_id=f183a0cf63a247ceb13603576e1edb17, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:45.053987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1420847-87ec-4948-b280-ffc984f65618 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:45.065012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 857/1691] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:45 2022] PUT /identity/v3/projects/89196dd924314a9d86d11e7fc1064058/users/f183a0cf63a247ceb13603576e1edb17/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:45.077976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de22aa-4063-4632-8944-cf4700353bb8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:45.078268 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de22aa-4063-4632-8944-cf4700353bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:45.078589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de22aa-4063-4632-8944-cf4700353bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:45.086798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:45.109099 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-30de22aa-4063-4632-8944-cf4700353bb8 None None] MFA Rules not processed for user `f183a0cf63a247ceb13603576e1edb17`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:45.110971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-30de22aa-4063-4632-8944-cf4700353bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:45.179745 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 835/1692] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:45 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:45.191224 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1420847-87ec-4948-b280-ffc984f65618 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:45.192925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d1420847-87ec-4948-b280-ffc984f65618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:45.253280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1420847-87ec-4948-b280-ffc984f65618 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:45.256935 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6636589-8d5b-4444-977a-6494112bdc34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:45.257567 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6636589-8d5b-4444-977a-6494112bdc34 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:45.257845 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6636589-8d5b-4444-977a-6494112bdc34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:45.258115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6636589-8d5b-4444-977a-6494112bdc34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:45.267597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6636589-8d5b-4444-977a-6494112bdc34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:45.269037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6636589-8d5b-4444-977a-6494112bdc34 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:45.278383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 858/1693] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:45 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:45.416983 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de22aa-4063-4632-8944-cf4700353bb8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:45.479965 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de22aa-4063-4632-8944-cf4700353bb8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:45.484465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:45.485371 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:45.485656 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:45.485936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] PATH_INFO: `/v3/users/ab0ca7470d7a47d7a8eb72c1d5bedf2a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:45.487061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0ca7470d7a47d7a8eb72c1d5bedf2a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:45.488988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:45.649377 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0ca7470d7a47d7a8eb72c1d5bedf2a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:45.650051 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0ca7470d7a47d7a8eb72c1d5bedf2a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:45.657171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0ca7470d7a47d7a8eb72c1d5bedf2a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:45.657935 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0ca7470d7a47d7a8eb72c1d5bedf2a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:45.715187 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0ca7470d7a47d7a8eb72c1d5bedf2a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:45.732821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 836/1694] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:45 2022] DELETE /identity/v3/users/ab0ca7470d7a47d7a8eb72c1d5bedf2a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:45.737781 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2282-06a6-495d-8c4c-f87f43719f5c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:45.738103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2282-06a6-495d-8c4c-f87f43719f5c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:45.738413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2282-06a6-495d-8c4c-f87f43719f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:45.758024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0c6a2282-06a6-495d-8c4c-f87f43719f5c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:45.759222 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0c6a2282-06a6-495d-8c4c-f87f43719f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:45.817339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 859/1695] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:45.827348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:45.829380 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:45.840157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:45.886916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd640bd-139d-46ba-ba46-85a3b00f4c74 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:45.891735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9486e-baac-4eb4-8b75-acf3c4f68d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:45.892473 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9486e-baac-4eb4-8b75-acf3c4f68d66 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:45.892792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9486e-baac-4eb4-8b75-acf3c4f68d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:45.893067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9486e-baac-4eb4-8b75-acf3c4f68d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:45.902236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9486e-baac-4eb4-8b75-acf3c4f68d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:45.903803 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9486e-baac-4eb4-8b75-acf3c4f68d66 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:45.914558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 837/1696] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:46.397977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a2282-06a6-495d-8c4c-f87f43719f5c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:46.399485 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0c6a2282-06a6-495d-8c4c-f87f43719f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:46.412983 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a2282-06a6-495d-8c4c-f87f43719f5c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:46.418308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:46.419037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:46.421269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:46.421269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] PATH_INFO: `/v3/projects/386dd306d30b4b61b3335c22d857a87f` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:46.421269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=386dd306d30b4b61b3335c22d857a87f)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:46.423657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:46.557854 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '386dd306d30b4b61b3335c22d857a87f', 'request_id': 'req-f8bc6610-ea74-4b73-9878-5441601a78c6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.635295 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '386dd306d30b4b61b3335c22d857a87f', 'request_id': 'req-f8bc6610-ea74-4b73-9878-5441601a78c6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.642370 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '386dd306d30b4b61b3335c22d857a87f', 'request_id': 'req-f8bc6610-ea74-4b73-9878-5441601a78c6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.653040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] The token cache is being invalidate because project 386dd306d30b4b61b3335c22d857a87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:46.653364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.653995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.655488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 860/1697] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:46 2022] DELETE /identity/v3/projects/386dd306d30b4b61b3335c22d857a87f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:46.660341 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9486e-baac-4eb4-8b75-acf3c4f68d66 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:46.670267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9486e-baac-4eb4-8b75-acf3c4f68d66 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:46.733816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:46.734454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:46.735241 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:46.735516 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] PATH_INFO: `/v3/users/f183a0cf63a247ceb13603576e1edb17` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:46.736652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f183a0cf63a247ceb13603576e1edb17)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:46.738614 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:46.862035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f183a0cf63a247ceb13603576e1edb17'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.862782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f183a0cf63a247ceb13603576e1edb17'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.870271 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f183a0cf63a247ceb13603576e1edb17'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.870887 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f183a0cf63a247ceb13603576e1edb17'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.932983 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f183a0cf63a247ceb13603576e1edb17'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:46.951826 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 838/1698] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:46 2022] DELETE /identity/v3/users/f183a0cf63a247ceb13603576e1edb17 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:47.381322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:47.383164 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:47.393357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:47.396326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:47.399124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:47.451488 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:47.473002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc6610-ea74-4b73-9878-5441601a78c6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:47.488264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2795b-be26-417a-a8b6-27a6d707525e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:47.494610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:47.495319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:47.495596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:47.495879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] PATH_INFO: `/v3/projects/89196dd924314a9d86d11e7fc1064058` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:47.497729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89196dd924314a9d86d11e7fc1064058)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:47.499650 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:47.531054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341b58a-7ef3-4083-a398-d5bec8f150a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:47.532340 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341b58a-7ef3-4083-a398-d5bec8f150a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:47.532616 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341b58a-7ef3-4083-a398-d5bec8f150a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:47.533064 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341b58a-7ef3-4083-a398-d5bec8f150a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:47.534901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5341b58a-7ef3-4083-a398-d5bec8f150a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:47.599331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341b58a-7ef3-4083-a398-d5bec8f150a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:47.600824 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341b58a-7ef3-4083-a398-d5bec8f150a9 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:47.608158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89196dd924314a9d86d11e7fc1064058', 'request_id': 'req-95d38caf-a087-4fbf-b016-0b7a1ce56a04'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:47.611111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 861/1699] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:47.615160 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89196dd924314a9d86d11e7fc1064058', 'request_id': 'req-95d38caf-a087-4fbf-b016-0b7a1ce56a04'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:47.686959 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89196dd924314a9d86d11e7fc1064058', 'request_id': 'req-95d38caf-a087-4fbf-b016-0b7a1ce56a04'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:47.698479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] The token cache is being invalidate because project 89196dd924314a9d86d11e7fc1064058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:47.698769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:47.699409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95d38caf-a087-4fbf-b016-0b7a1ce56a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:47.700924 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 839/1700] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:47 2022] DELETE /identity/v3/projects/89196dd924314a9d86d11e7fc1064058 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:47.707026 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc409e-4c6b-4a33-bd9d-cfd2b230225c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:47.707318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc409e-4c6b-4a33-bd9d-cfd2b230225c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:47.707600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc409e-4c6b-4a33-bd9d-cfd2b230225c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:47.727128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7adc409e-4c6b-4a33-bd9d-cfd2b230225c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:47.728327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7adc409e-4c6b-4a33-bd9d-cfd2b230225c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:47.798752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 862/1701] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:47.806562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed37ac-721b-4ccd-968e-fe5b71a30920 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:47.806853 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed37ac-721b-4ccd-968e-fe5b71a30920 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:47.807164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed37ac-721b-4ccd-968e-fe5b71a30920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:47.822116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc409e-4c6b-4a33-bd9d-cfd2b230225c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:47.824101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7adc409e-4c6b-4a33-bd9d-cfd2b230225c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:47.828948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-67ed37ac-721b-4ccd-968e-fe5b71a30920 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:47.830490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-67ed37ac-721b-4ccd-968e-fe5b71a30920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:47.836831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc409e-4c6b-4a33-bd9d-cfd2b230225c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:47.839008 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 840/1702] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:47.841730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:47.842705 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:47.842988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:47.843081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed37ac-721b-4ccd-968e-fe5b71a30920 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:47.843290 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] PATH_INFO: `/v3/users/014f45dfa52942d0901bcc300af09c48` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:47.844506 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=014f45dfa52942d0901bcc300af09c48)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:47.846493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:47.857026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed37ac-721b-4ccd-968e-fe5b71a30920 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:47.857600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d09da-4a71-40b7-9cc8-e1de26500347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:47.858052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:47.858234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d09da-4a71-40b7-9cc8-e1de26500347 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:47.858686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d09da-4a71-40b7-9cc8-e1de26500347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:47.858966 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d09da-4a71-40b7-9cc8-e1de26500347 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:47.859629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d09da-4a71-40b7-9cc8-e1de26500347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:47.860021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d09da-4a71-40b7-9cc8-e1de26500347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:47.861671 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d09da-4a71-40b7-9cc8-e1de26500347 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:47.873943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 841/1703] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:04:47.879028 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d09da-4a71-40b7-9cc8-e1de26500347 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:47.888922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d09da-4a71-40b7-9cc8-e1de26500347 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:47.894383 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b7b93-7815-4f84-b166-8cd8541f05b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:47.894971 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b7b93-7815-4f84-b166-8cd8541f05b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:47.895396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b7b93-7815-4f84-b166-8cd8541f05b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:47.895681 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b7b93-7815-4f84-b166-8cd8541f05b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:47.896307 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b7b93-7815-4f84-b166-8cd8541f05b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:47.898338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b7b93-7815-4f84-b166-8cd8541f05b1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:47.979895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 842/1704] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:47 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:47.983864 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b7b93-7815-4f84-b166-8cd8541f05b1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:48.052933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014f45dfa52942d0901bcc300af09c48'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:48.059930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b7b93-7815-4f84-b166-8cd8541f05b1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:48.061023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014f45dfa52942d0901bcc300af09c48'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:48.061771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014f45dfa52942d0901bcc300af09c48'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:48.062450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014f45dfa52942d0901bcc300af09c48'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:48.067424 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c9ae-fa4b-4bad-852b-500d437c50b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:48.067424 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c9ae-fa4b-4bad-852b-500d437c50b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:48.067628 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c9ae-fa4b-4bad-852b-500d437c50b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:48.067835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c9ae-fa4b-4bad-852b-500d437c50b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:48.068702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0c9ae-fa4b-4bad-852b-500d437c50b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:48.070671 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0c9ae-fa4b-4bad-852b-500d437c50b1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:48.173161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8b7c487-0a08-4b81-9681-9b6ecfe2f602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014f45dfa52942d0901bcc300af09c48'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:48.180407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 843/1705] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:47 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:48.185514 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c9ae-fa4b-4bad-852b-500d437c50b1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:48.188236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3ee0c9ae-fa4b-4bad-852b-500d437c50b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:48.195121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 863/1706] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:47 2022] DELETE /identity/v3/users/014f45dfa52942d0901bcc300af09c48 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:48.199759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f8bf0-1acd-4240-8097-e7b5fbed91ce None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:48.200053 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f8bf0-1acd-4240-8097-e7b5fbed91ce None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:48.200328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f8bf0-1acd-4240-8097-e7b5fbed91ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:48.202005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c9ae-fa4b-4bad-852b-500d437c50b1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:48.206976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:48.220141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d05f8bf0-1acd-4240-8097-e7b5fbed91ce None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:48.221434 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d05f8bf0-1acd-4240-8097-e7b5fbed91ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:48.260739 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb9e72-9626-4762-b480-29d424c816ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:48.261399 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb9e72-9626-4762-b480-29d424c816ea admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:48.261683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb9e72-9626-4762-b480-29d424c816ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:48.262245 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb9e72-9626-4762-b480-29d424c816ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:48.262605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb9e72-9626-4762-b480-29d424c816ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:48.264227 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb9e72-9626-4762-b480-29d424c816ea admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:48.274563 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 844/1707] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:48.278511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb9e72-9626-4762-b480-29d424c816ea admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:48.290644 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 864/1708] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:48.290816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb9e72-9626-4762-b480-29d424c816ea admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:48.294377 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae2c0c-7d23-4ad3-b77d-5cb455998dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:48.294986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae2c0c-7d23-4ad3-b77d-5cb455998dee admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:48.295266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae2c0c-7d23-4ad3-b77d-5cb455998dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:48.295588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae2c0c-7d23-4ad3-b77d-5cb455998dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:48.296252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae2c0c-7d23-4ad3-b77d-5cb455998dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:48.298376 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f8bf0-1acd-4240-8097-e7b5fbed91ce None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:48.299941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d05f8bf0-1acd-4240-8097-e7b5fbed91ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:48.302297 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae2c0c-7d23-4ad3-b77d-5cb455998dee admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:48.317292 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 845/1709] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:48.318256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae2c0c-7d23-4ad3-b77d-5cb455998dee admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:48.328430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae2c0c-7d23-4ad3-b77d-5cb455998dee admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:48.333475 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769b2ec-a1be-4009-8d51-512c66c9f59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:48.334143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769b2ec-a1be-4009-8d51-512c66c9f59a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:48.334429 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769b2ec-a1be-4009-8d51-512c66c9f59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:48.334709 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769b2ec-a1be-4009-8d51-512c66c9f59a admin admin] PATH_INFO: `/v3/projects/5a9af80c7f3043e49c3f7da282f77c49/users/4037a903acb945d78ef4cd67eab00bfc/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:48.349879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769b2ec-a1be-4009-8d51-512c66c9f59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9af80c7f3043e49c3f7da282f77c49, user_id=4037a903acb945d78ef4cd67eab00bfc, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:48.352551 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769b2ec-a1be-4009-8d51-512c66c9f59a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:48.357209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f8bf0-1acd-4240-8097-e7b5fbed91ce None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:48.361672 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cff4e4-b39b-4d11-b822-71c6393065ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:48.362289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cff4e4-b39b-4d11-b822-71c6393065ae service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:48.362571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cff4e4-b39b-4d11-b822-71c6393065ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:48.362850 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cff4e4-b39b-4d11-b822-71c6393065ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:48.372152 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cff4e4-b39b-4d11-b822-71c6393065ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:48.373640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cff4e4-b39b-4d11-b822-71c6393065ae service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:48.383121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 865/1710] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:48.447183 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 846/1711] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:48 2022] PUT /identity/v3/projects/5a9af80c7f3043e49c3f7da282f77c49/users/4037a903acb945d78ef4cd67eab00bfc/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:48.459774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc5270-9ff4-48bb-90d1-84e6b42a8fe6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:48.460072 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc5270-9ff4-48bb-90d1-84e6b42a8fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:48.460354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc5270-9ff4-48bb-90d1-84e6b42a8fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:48.492789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-60cc5270-9ff4-48bb-90d1-84e6b42a8fe6 None None] MFA Rules not processed for user `4037a903acb945d78ef4cd67eab00bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:48.494061 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-60cc5270-9ff4-48bb-90d1-84e6b42a8fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:48.517235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:48.582511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 866/1712] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:48.587087 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769b2ec-a1be-4009-8d51-512c66c9f59a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:48.707347 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769b2ec-a1be-4009-8d51-512c66c9f59a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:48.713269 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945e178-202e-4970-bafa-4ea3c5b0dfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:48.714164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945e178-202e-4970-bafa-4ea3c5b0dfbc admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:48.714809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945e178-202e-4970-bafa-4ea3c5b0dfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:48.715298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945e178-202e-4970-bafa-4ea3c5b0dfbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:48.716115 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945e178-202e-4970-bafa-4ea3c5b0dfbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:48.718645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945e178-202e-4970-bafa-4ea3c5b0dfbc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:48.849665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 847/1713] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:48 2022] POST /identity/v3/projects => generated 365 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:48.853798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc5270-9ff4-48bb-90d1-84e6b42a8fe6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:48.864526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc5270-9ff4-48bb-90d1-84e6b42a8fe6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:48.888099 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945e178-202e-4970-bafa-4ea3c5b0dfbc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:48.892316 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4945e178-202e-4970-bafa-4ea3c5b0dfbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:48.944281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a11fb-1a4e-48a5-9dbc-366fd8700437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:48.944815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a11fb-1a4e-48a5-9dbc-366fd8700437 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:48.945101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a11fb-1a4e-48a5-9dbc-366fd8700437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:48.945403 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a11fb-1a4e-48a5-9dbc-366fd8700437 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:48.946102 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a11fb-1a4e-48a5-9dbc-366fd8700437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:48.948052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a11fb-1a4e-48a5-9dbc-366fd8700437 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:48.989010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:48.994517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945e178-202e-4970-bafa-4ea3c5b0dfbc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:48.999717 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:49.000464 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.000740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.001011 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] PATH_INFO: `/v3/projects/3fe96e8d8ea44e0b9e1dcc0efbb83250` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.002161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe96e8d8ea44e0b9e1dcc0efbb83250)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:49.004192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:49.039424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 867/1714] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:48 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:49.040872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a11fb-1a4e-48a5-9dbc-366fd8700437 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:49.051258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a11fb-1a4e-48a5-9dbc-366fd8700437 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:49.056127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d34c4-d8c1-433f-bb31-e7872798a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:49.057428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d34c4-d8c1-433f-bb31-e7872798a30d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.057706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d34c4-d8c1-433f-bb31-e7872798a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.058000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d34c4-d8c1-433f-bb31-e7872798a30d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.058641 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d34c4-d8c1-433f-bb31-e7872798a30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:49.060246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d34c4-d8c1-433f-bb31-e7872798a30d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:49.073230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 868/1715] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:49.075929 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d34c4-d8c1-433f-bb31-e7872798a30d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:49.086677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d34c4-d8c1-433f-bb31-e7872798a30d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:49.091861 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cc016-cb54-40a5-88eb-93a3609daa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:49.092708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cc016-cb54-40a5-88eb-93a3609daa54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.093089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cc016-cb54-40a5-88eb-93a3609daa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.093489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cc016-cb54-40a5-88eb-93a3609daa54 admin admin] PATH_INFO: `/v3/projects/47ae602122874952aa27755e352634ba/users/9e95006e219e40f3bd2825bc6bf6c95b/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.115167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cc016-cb54-40a5-88eb-93a3609daa54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ae602122874952aa27755e352634ba, user_id=9e95006e219e40f3bd2825bc6bf6c95b, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:49.118513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cc016-cb54-40a5-88eb-93a3609daa54 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:49.126877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe96e8d8ea44e0b9e1dcc0efbb83250', 'request_id': 'req-627ca596-21a9-443f-b16b-26bea3cc0fc1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.134720 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe96e8d8ea44e0b9e1dcc0efbb83250', 'request_id': 'req-627ca596-21a9-443f-b16b-26bea3cc0fc1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.195927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe96e8d8ea44e0b9e1dcc0efbb83250', 'request_id': 'req-627ca596-21a9-443f-b16b-26bea3cc0fc1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.203161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 869/1716] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:49 2022] PUT /identity/v3/projects/47ae602122874952aa27755e352634ba/users/9e95006e219e40f3bd2825bc6bf6c95b/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:49.208038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cc016-cb54-40a5-88eb-93a3609daa54 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:49.209230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] The token cache is being invalidate because project 3fe96e8d8ea44e0b9e1dcc0efbb83250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:49.209698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.210533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.212450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 848/1717] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:48 2022] DELETE /identity/v3/projects/3fe96e8d8ea44e0b9e1dcc0efbb83250 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:49.216621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:49.218592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:49.286798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cc016-cb54-40a5-88eb-93a3609daa54 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:49.292281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39a676-845d-436c-a741-f7c7e6419c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:49.292956 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39a676-845d-436c-a741-f7c7e6419c30 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.293271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39a676-845d-436c-a741-f7c7e6419c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.293548 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39a676-845d-436c-a741-f7c7e6419c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.294197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39a676-845d-436c-a741-f7c7e6419c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:49.295841 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39a676-845d-436c-a741-f7c7e6419c30 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:49.301644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ca596-21a9-443f-b16b-26bea3cc0fc1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:49.307925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 870/1718] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:04:49.309629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:49.310311 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.310607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.310877 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] PATH_INFO: `/v3/users/387e74f1de0643a6a47f153857496026` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.312177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=387e74f1de0643a6a47f153857496026)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:49.313624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39a676-845d-436c-a741-f7c7e6419c30 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:49.315379 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:49.316015 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fa39a676-845d-436c-a741-f7c7e6419c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:49.331141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39a676-845d-436c-a741-f7c7e6419c30 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:49.336363 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c697d-9999-4163-bdaa-71b85b8523d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:49.337171 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c697d-9999-4163-bdaa-71b85b8523d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.337545 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c697d-9999-4163-bdaa-71b85b8523d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.337889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c697d-9999-4163-bdaa-71b85b8523d6 admin admin] PATH_INFO: `/v3/projects/47ae602122874952aa27755e352634ba/users/9e95006e219e40f3bd2825bc6bf6c95b/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.339935 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c697d-9999-4163-bdaa-71b85b8523d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ae602122874952aa27755e352634ba, user_id=9e95006e219e40f3bd2825bc6bf6c95b, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:49.346076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c697d-9999-4163-bdaa-71b85b8523d6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:49.352811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:49.358099 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 871/1719] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:49 2022] PUT /identity/v3/projects/47ae602122874952aa27755e352634ba/users/9e95006e219e40f3bd2825bc6bf6c95b/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:49.369622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cbff3-8847-4995-bea6-dca1c8257c5f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.369909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cbff3-8847-4995-bea6-dca1c8257c5f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.370183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cbff3-8847-4995-bea6-dca1c8257c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.399763 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5c5cbff3-8847-4995-bea6-dca1c8257c5f None None] MFA Rules not processed for user `9e95006e219e40f3bd2825bc6bf6c95b`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:49.403067 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5c5cbff3-8847-4995-bea6-dca1c8257c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:49.468102 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 872/1720] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:49.469138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cbff3-8847-4995-bea6-dca1c8257c5f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:49.470707 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5c5cbff3-8847-4995-bea6-dca1c8257c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:49.495272 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '387e74f1de0643a6a47f153857496026'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.500876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '387e74f1de0643a6a47f153857496026'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.504157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '387e74f1de0643a6a47f153857496026'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.504879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '387e74f1de0643a6a47f153857496026'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.527192 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cbff3-8847-4995-bea6-dca1c8257c5f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:49.530893 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:49.531667 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.532059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.532325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.534131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:49.549874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '387e74f1de0643a6a47f153857496026'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:49.575055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 849/1721] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:49 2022] DELETE /identity/v3/users/387e74f1de0643a6a47f153857496026 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:49.576076 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:49.577432 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:49.583900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:49.608818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:49.611103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:49.622623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 873/1722] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:49.641873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de605dc-d172-488b-bae7-c763e887f8c0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:49.646989 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:49.647741 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.648363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.649136 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.652727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:49.717226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:49.718733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:49.729420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 850/1723] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:49.948183 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:49.949612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:49.960523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76317d6b-99fa-4e18-b963-430f53791bd7 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:49.964132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f5869-baee-4b47-8c04-e76d6fa6255c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:49.964734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f5869-baee-4b47-8c04-e76d6fa6255c service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:49.965009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f5869-baee-4b47-8c04-e76d6fa6255c service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:49.965384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f5869-baee-4b47-8c04-e76d6fa6255c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:49.967030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-529f5869-baee-4b47-8c04-e76d6fa6255c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:49.998722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:50.009307 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:50.010767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:50.030534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f5869-baee-4b47-8c04-e76d6fa6255c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:50.033527 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f5869-baee-4b47-8c04-e76d6fa6255c service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:50.043345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 874/1724] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:50.082716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91f595-08f5-4246-9646-3b76180412fa service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:50.087470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:50.088108 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:50.088391 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:50.088668 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] PATH_INFO: `/v3/projects/e75152bddeb5411899e496586ef6907b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:50.089819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e75152bddeb5411899e496586ef6907b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:50.091737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:50.200244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e75152bddeb5411899e496586ef6907b', 'request_id': 'req-6e471323-995b-4230-bb24-0d2cfac7c203'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.208131 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e75152bddeb5411899e496586ef6907b', 'request_id': 'req-6e471323-995b-4230-bb24-0d2cfac7c203'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.210742 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:50.274046 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e75152bddeb5411899e496586ef6907b', 'request_id': 'req-6e471323-995b-4230-bb24-0d2cfac7c203'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.286242 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] The token cache is being invalidate because project e75152bddeb5411899e496586ef6907b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:50.286530 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.287177 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.288648 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 851/1725] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:50 2022] DELETE /identity/v3/projects/e75152bddeb5411899e496586ef6907b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:50.292588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f5869-baee-4b47-8c04-e76d6fa6255c service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:50.303788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f5869-baee-4b47-8c04-e76d6fa6255c service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:50.361501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:50.362141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:50.362427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:50.362707 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] PATH_INFO: `/v3/users/635927a4f044413cacf10a41cf798683` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:50.363833 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=635927a4f044413cacf10a41cf798683)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:50.365844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:50.530346 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '635927a4f044413cacf10a41cf798683'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.538192 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '635927a4f044413cacf10a41cf798683'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.538892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '635927a4f044413cacf10a41cf798683'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.539498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '635927a4f044413cacf10a41cf798683'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.623629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-14344330-fa7c-4328-a58b-eeddfe553962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '635927a4f044413cacf10a41cf798683'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:50.648200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 875/1726] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:50 2022] DELETE /identity/v3/users/635927a4f044413cacf10a41cf798683 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:51.243350 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:51.245466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:51.259253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e471323-995b-4230-bb24-0d2cfac7c203 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:51.319715 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:51.320359 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:51.320641 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:51.320916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] PATH_INFO: `/v3/projects/50ee57951ead4a7b8a86bda7a7e40877` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:51.322069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] RBAC: Authorizing `identity:delete_project(project_id=50ee57951ead4a7b8a86bda7a7e40877)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:51.324017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:51.428759 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50ee57951ead4a7b8a86bda7a7e40877', 'request_id': 'req-6a398ed0-ca97-4a28-ae31-134e0b23bfac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:51.435361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50ee57951ead4a7b8a86bda7a7e40877', 'request_id': 'req-6a398ed0-ca97-4a28-ae31-134e0b23bfac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:51.527039 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50ee57951ead4a7b8a86bda7a7e40877', 'request_id': 'req-6a398ed0-ca97-4a28-ae31-134e0b23bfac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:51.543996 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] The token cache is being invalidate because project 50ee57951ead4a7b8a86bda7a7e40877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:51.544323 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:51.544980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6a398ed0-ca97-4a28-ae31-134e0b23bfac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:51.546540 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 852/1727] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:51 2022] DELETE /identity/v3/projects/50ee57951ead4a7b8a86bda7a7e40877 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:51.552038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd7df8-a111-46a3-9858-4858eab07df1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:51.552788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd7df8-a111-46a3-9858-4858eab07df1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:51.552866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd7df8-a111-46a3-9858-4858eab07df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:51.572912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7fbd7df8-a111-46a3-9858-4858eab07df1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:51.574157 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7fbd7df8-a111-46a3-9858-4858eab07df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:51.620634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:51.634875 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 876/1728] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:51.644214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c66ef-9a6c-4d2d-8214-a820acc8f408 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:51.644507 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c66ef-9a6c-4d2d-8214-a820acc8f408 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:51.644790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c66ef-9a6c-4d2d-8214-a820acc8f408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:51.666865 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-427c66ef-9a6c-4d2d-8214-a820acc8f408 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:51.668086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-427c66ef-9a6c-4d2d-8214-a820acc8f408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:51.676426 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 853/1729] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:51.681047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd7df8-a111-46a3-9858-4858eab07df1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:51.690641 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd7df8-a111-46a3-9858-4858eab07df1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:51.695634 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00139f-8775-49d1-a6e9-5a32d7490ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:51.696042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00139f-8775-49d1-a6e9-5a32d7490ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:51.696333 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00139f-8775-49d1-a6e9-5a32d7490ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:51.696633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00139f-8775-49d1-a6e9-5a32d7490ba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:51.697296 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00139f-8775-49d1-a6e9-5a32d7490ba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:51.697605 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00139f-8775-49d1-a6e9-5a32d7490ba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:51.699214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00139f-8775-49d1-a6e9-5a32d7490ba7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:51.711404 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 877/1730] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:51.718231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c66ef-9a6c-4d2d-8214-a820acc8f408 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:51.728684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c66ef-9a6c-4d2d-8214-a820acc8f408 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:51.733436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d0d15-a801-4a98-89a7-3aeaae99727f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:51.734042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d0d15-a801-4a98-89a7-3aeaae99727f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:51.734324 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d0d15-a801-4a98-89a7-3aeaae99727f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:51.734692 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d0d15-a801-4a98-89a7-3aeaae99727f admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:51.735618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d0d15-a801-4a98-89a7-3aeaae99727f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:51.737569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d0d15-a801-4a98-89a7-3aeaae99727f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:51.802573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 854/1731] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:51 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:51.806915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00139f-8775-49d1-a6e9-5a32d7490ba7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:51.816614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00139f-8775-49d1-a6e9-5a32d7490ba7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:51.871980 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09569d4b-49c1-4aaf-81a9-e4d0945ad3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:51.872666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09569d4b-49c1-4aaf-81a9-e4d0945ad3ca admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:51.872947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09569d4b-49c1-4aaf-81a9-e4d0945ad3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:51.873254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09569d4b-49c1-4aaf-81a9-e4d0945ad3ca admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:51.873890 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09569d4b-49c1-4aaf-81a9-e4d0945ad3ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:51.876938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09569d4b-49c1-4aaf-81a9-e4d0945ad3ca admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:51.953938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 878/1732] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:51 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:51.955913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d0d15-a801-4a98-89a7-3aeaae99727f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:51.966700 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d0d15-a801-4a98-89a7-3aeaae99727f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:51.972691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61225738-174e-4181-84e6-1cfef5fab75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:51.973370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61225738-174e-4181-84e6-1cfef5fab75f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:51.973655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61225738-174e-4181-84e6-1cfef5fab75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:51.974204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61225738-174e-4181-84e6-1cfef5fab75f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:51.974945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61225738-174e-4181-84e6-1cfef5fab75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:51.976588 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61225738-174e-4181-84e6-1cfef5fab75f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:51.987621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 855/1733] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:51.991741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09569d4b-49c1-4aaf-81a9-e4d0945ad3ca admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:52.002806 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09569d4b-49c1-4aaf-81a9-e4d0945ad3ca admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:52.007532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561e111-d715-43ba-8c34-837fe2f4cec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:52.008143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561e111-d715-43ba-8c34-837fe2f4cec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:52.008428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561e111-d715-43ba-8c34-837fe2f4cec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:52.008703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561e111-d715-43ba-8c34-837fe2f4cec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:52.009389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561e111-d715-43ba-8c34-837fe2f4cec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:52.011032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561e111-d715-43ba-8c34-837fe2f4cec7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:52.022729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 879/1734] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:52.026894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61225738-174e-4181-84e6-1cfef5fab75f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:52.037437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61225738-174e-4181-84e6-1cfef5fab75f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:52.042445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446267f-7fbd-430a-8c14-5fdadc8f5e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:52.043099 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446267f-7fbd-430a-8c14-5fdadc8f5e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:52.045694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446267f-7fbd-430a-8c14-5fdadc8f5e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:52.045694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446267f-7fbd-430a-8c14-5fdadc8f5e5f admin admin] PATH_INFO: `/v3/projects/e1a1d93e23424aef9e369d5aaaff3405/users/7901f618fe9f448086b2ed512fe243f0/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:52.059974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446267f-7fbd-430a-8c14-5fdadc8f5e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a1d93e23424aef9e369d5aaaff3405, user_id=7901f618fe9f448086b2ed512fe243f0, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:52.062990 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446267f-7fbd-430a-8c14-5fdadc8f5e5f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:52.182221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 856/1735] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:52 2022] PUT /identity/v3/projects/e1a1d93e23424aef9e369d5aaaff3405/users/7901f618fe9f448086b2ed512fe243f0/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:52.193451 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11b8d2-0b15-4d89-95a5-22b24bbcd88a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:52.193692 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11b8d2-0b15-4d89-95a5-22b24bbcd88a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:52.194194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11b8d2-0b15-4d89-95a5-22b24bbcd88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:52.228062 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-db11b8d2-0b15-4d89-95a5-22b24bbcd88a None None] MFA Rules not processed for user `7901f618fe9f448086b2ed512fe243f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:52.229477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db11b8d2-0b15-4d89-95a5-22b24bbcd88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:52.302563 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 880/1736] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:52.314381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446267f-7fbd-430a-8c14-5fdadc8f5e5f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:52.316961 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9446267f-7fbd-430a-8c14-5fdadc8f5e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:52.334416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:52.392548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446267f-7fbd-430a-8c14-5fdadc8f5e5f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:52.398104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9149ccb-4e08-4d17-81a7-faec13de3b70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:52.398890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9149ccb-4e08-4d17-81a7-faec13de3b70 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:52.399324 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9149ccb-4e08-4d17-81a7-faec13de3b70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:52.399627 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9149ccb-4e08-4d17-81a7-faec13de3b70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:52.410088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9149ccb-4e08-4d17-81a7-faec13de3b70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:52.411679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9149ccb-4e08-4d17-81a7-faec13de3b70 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:52.424908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 857/1737] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:54.740816 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11b8d2-0b15-4d89-95a5-22b24bbcd88a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:54.742284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db11b8d2-0b15-4d89-95a5-22b24bbcd88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:54.801965 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:54.826172 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11b8d2-0b15-4d89-95a5-22b24bbcd88a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:54.833936 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:54.834512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:54.834802 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:54.835081 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] PATH_INFO: `/v3/users/e8a6ac08778e49f79a78c7faf537c3c9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:54.837217 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8a6ac08778e49f79a78c7faf537c3c9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:54.839726 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:55.034110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8a6ac08778e49f79a78c7faf537c3c9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:55.043199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8a6ac08778e49f79a78c7faf537c3c9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:55.044018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8a6ac08778e49f79a78c7faf537c3c9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:55.044668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8a6ac08778e49f79a78c7faf537c3c9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:55.155538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8a6ac08778e49f79a78c7faf537c3c9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:55.176523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 881/1738] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:54 2022] DELETE /identity/v3/users/e8a6ac08778e49f79a78c7faf537c3c9 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:55.184504 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf4bab-6bf0-4b83-bac9-5737c451165f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:55.185009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf4bab-6bf0-4b83-bac9-5737c451165f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:55.185100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf4bab-6bf0-4b83-bac9-5737c451165f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:55.204505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7fcf4bab-6bf0-4b83-bac9-5737c451165f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:55.205884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7fcf4bab-6bf0-4b83-bac9-5737c451165f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:55.270034 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 858/1739] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:55.280078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:55.281555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:55.352725 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d97f49-4c9b-4341-ab70-fd1e23014033 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:55.356657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e7aca-e2f1-4f66-9acf-bdf04adad9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:55.359992 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e7aca-e2f1-4f66-9acf-bdf04adad9fb service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:55.359992 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e7aca-e2f1-4f66-9acf-bdf04adad9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:55.359992 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e7aca-e2f1-4f66-9acf-bdf04adad9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:55.367974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e7aca-e2f1-4f66-9acf-bdf04adad9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:55.369449 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e7aca-e2f1-4f66-9acf-bdf04adad9fb service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:55.380039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 882/1740] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:55.814564 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf4bab-6bf0-4b83-bac9-5737c451165f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:55.816485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7fcf4bab-6bf0-4b83-bac9-5737c451165f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:55.840892 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf4bab-6bf0-4b83-bac9-5737c451165f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:55.848172 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:55.849075 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:55.849483 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:55.849871 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] PATH_INFO: `/v3/projects/9e47d86b67b24e0fa8b59d7b2822c0ae` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:55.851768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e47d86b67b24e0fa8b59d7b2822c0ae)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:55.855687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:55.979817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e47d86b67b24e0fa8b59d7b2822c0ae', 'request_id': 'req-8734d7e8-c158-4777-8e31-097cc7141150'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:55.986124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e47d86b67b24e0fa8b59d7b2822c0ae', 'request_id': 'req-8734d7e8-c158-4777-8e31-097cc7141150'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:56.064039 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e47d86b67b24e0fa8b59d7b2822c0ae', 'request_id': 'req-8734d7e8-c158-4777-8e31-097cc7141150'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:56.075133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] The token cache is being invalidate because project 9e47d86b67b24e0fa8b59d7b2822c0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:56.075422 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:56.076043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:56.077784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 859/1741] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:55 2022] DELETE /identity/v3/projects/9e47d86b67b24e0fa8b59d7b2822c0ae => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:56.083110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e7aca-e2f1-4f66-9acf-bdf04adad9fb service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:56.093054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e7aca-e2f1-4f66-9acf-bdf04adad9fb service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:56.159899 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:56.160610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:56.161012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:56.161383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] PATH_INFO: `/v3/users/d62988bfa6454a97a09b52a314292145` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:56.162614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62988bfa6454a97a09b52a314292145)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:56.164689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:56.176086 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:56.330907 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62988bfa6454a97a09b52a314292145'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:56.340656 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62988bfa6454a97a09b52a314292145'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:56.341411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62988bfa6454a97a09b52a314292145'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:56.342030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62988bfa6454a97a09b52a314292145'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:56.436261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62988bfa6454a97a09b52a314292145'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:56.467863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 883/1742] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:56 2022] DELETE /identity/v3/users/d62988bfa6454a97a09b52a314292145 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:56.709693 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:56.712244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:56.731968 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734d7e8-c158-4777-8e31-097cc7141150 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:56.790550 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:56.819626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:56.820334 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:56.820680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:56.821048 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] PATH_INFO: `/v3/users/7901f618fe9f448086b2ed512fe243f0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:56.823104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7901f618fe9f448086b2ed512fe243f0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:56.826770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:57.004756 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:57.006806 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:57.034995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3daba-49eb-4e24-9bc0-248dc5ffebd8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:57.040909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:57.052502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:57.052894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:57.053159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] PATH_INFO: `/v3/projects/78d8f88e764e4867a7553741bf622ff3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:57.054711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d8f88e764e4867a7553741bf622ff3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:57.056553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:57.078943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7901f618fe9f448086b2ed512fe243f0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.079740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7901f618fe9f448086b2ed512fe243f0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.089089 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7901f618fe9f448086b2ed512fe243f0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.090354 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7901f618fe9f448086b2ed512fe243f0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.155495 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d70a8a1-43f4-40b2-ace5-c1e23414f145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7901f618fe9f448086b2ed512fe243f0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.191938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d8f88e764e4867a7553741bf622ff3', 'request_id': 'req-801d96e9-0a4f-4f21-977d-dd5128f79173'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.192492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 860/1743] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:56 2022] DELETE /identity/v3/users/7901f618fe9f448086b2ed512fe243f0 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:57.197244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e26e-1c7b-41c0-8b83-91d6aa84ffea None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:57.198161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e26e-1c7b-41c0-8b83-91d6aa84ffea None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:57.198519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e26e-1c7b-41c0-8b83-91d6aa84ffea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:57.220619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4061e26e-1c7b-41c0-8b83-91d6aa84ffea None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:57.223070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4061e26e-1c7b-41c0-8b83-91d6aa84ffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:57.252800 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d8f88e764e4867a7553741bf622ff3', 'request_id': 'req-801d96e9-0a4f-4f21-977d-dd5128f79173'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.261112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d8f88e764e4867a7553741bf622ff3', 'request_id': 'req-801d96e9-0a4f-4f21-977d-dd5128f79173'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.281943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] The token cache is being invalidate because project 78d8f88e764e4867a7553741bf622ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:57.282706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.285092 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.287528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 884/1744] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:57 2022] DELETE /identity/v3/projects/78d8f88e764e4867a7553741bf622ff3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:57.292851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:57.295472 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:57.309950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 861/1745] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:04:57.322579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061e26e-1c7b-41c0-8b83-91d6aa84ffea None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:57.324268 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4061e26e-1c7b-41c0-8b83-91d6aa84ffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:57.392271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d96e9-0a4f-4f21-977d-dd5128f79173 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:57.399231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061e26e-1c7b-41c0-8b83-91d6aa84ffea None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:57.400722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:57.401544 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:57.401990 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:57.402280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] PATH_INFO: `/v3/users/d4b72784459d41e1b6079cc361711eca` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:57.403457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4b72784459d41e1b6079cc361711eca)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:57.404249 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13496db-b15e-40a9-8f45-37c0c7629245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:57.405122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13496db-b15e-40a9-8f45-37c0c7629245 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:57.405540 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13496db-b15e-40a9-8f45-37c0c7629245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:57.405736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:57.406038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13496db-b15e-40a9-8f45-37c0c7629245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:57.420373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13496db-b15e-40a9-8f45-37c0c7629245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:57.424209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13496db-b15e-40a9-8f45-37c0c7629245 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:57.431543 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:57.436929 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 862/1746] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:04:57.438714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:57.628750 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4b72784459d41e1b6079cc361711eca'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.636640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4b72784459d41e1b6079cc361711eca'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.637384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4b72784459d41e1b6079cc361711eca'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.637988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4b72784459d41e1b6079cc361711eca'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.744911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4b72784459d41e1b6079cc361711eca'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:57.768341 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 885/1747] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:57 2022] DELETE /identity/v3/users/d4b72784459d41e1b6079cc361711eca => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:57.939822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13496db-b15e-40a9-8f45-37c0c7629245 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:57.942445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e13496db-b15e-40a9-8f45-37c0c7629245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:57.956790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13496db-b15e-40a9-8f45-37c0c7629245 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:58.057070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:58.057297 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:58.060914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:58.060914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] PATH_INFO: `/v3/projects/e1a1d93e23424aef9e369d5aaaff3405` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:58.060914 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a1d93e23424aef9e369d5aaaff3405)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:58.061519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:58.240964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a1d93e23424aef9e369d5aaaff3405', 'request_id': 'req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.254199 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a1d93e23424aef9e369d5aaaff3405', 'request_id': 'req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.323424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:58.325068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:58.342794 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab559bfb-7f83-4ffe-8e45-4e42d6a05061 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:58.367140 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a1d93e23424aef9e369d5aaaff3405', 'request_id': 'req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.383982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] The token cache is being invalidate because project e1a1d93e23424aef9e369d5aaaff3405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:58.384366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.385113 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.394631 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 863/1748] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:57 2022] DELETE /identity/v3/projects/e1a1d93e23424aef9e369d5aaaff3405 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:58.395385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:58.396788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:58.444000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:58.444976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:58.445525 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:58.446144 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] PATH_INFO: `/v3/projects/e06ed4a0c9ad40cfa80e9338d299154d` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:58.448362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e06ed4a0c9ad40cfa80e9338d299154d)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:58.451512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:58.479957 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb3fdf-7444-4a72-84a6-39c3fc9239e3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:58.486445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:58.487114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:58.487467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:58.487741 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] PATH_INFO: `/v3/users/4037a903acb945d78ef4cd67eab00bfc` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:58.489636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4037a903acb945d78ef4cd67eab00bfc)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:58.491931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:58.661032 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e06ed4a0c9ad40cfa80e9338d299154d', 'request_id': 'req-4d1d63c6-284c-4942-ab4d-8778b8e573fb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.726671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e06ed4a0c9ad40cfa80e9338d299154d', 'request_id': 'req-4d1d63c6-284c-4942-ab4d-8778b8e573fb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.727264 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:58.745747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e06ed4a0c9ad40cfa80e9338d299154d', 'request_id': 'req-4d1d63c6-284c-4942-ab4d-8778b8e573fb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.768263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4037a903acb945d78ef4cd67eab00bfc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.772064 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4037a903acb945d78ef4cd67eab00bfc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.783255 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] The token cache is being invalidate because project e06ed4a0c9ad40cfa80e9338d299154d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:04:58.783759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.785294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4d1d63c6-284c-4942-ab4d-8778b8e573fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.789239 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 886/1749] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:58 2022] DELETE /identity/v3/projects/e06ed4a0c9ad40cfa80e9338d299154d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:58.796060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4037a903acb945d78ef4cd67eab00bfc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.796292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dc865-4bde-452d-a087-e844def575f0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:58.796292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dc865-4bde-452d-a087-e844def575f0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:58.796292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dc865-4bde-452d-a087-e844def575f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:58.796687 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4037a903acb945d78ef4cd67eab00bfc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.828571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-105dc865-4bde-452d-a087-e844def575f0 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:58.837865 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-105dc865-4bde-452d-a087-e844def575f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:58.858558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:58.898327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db52758b-e2e5-4b4b-aac0-695b36563a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4037a903acb945d78ef4cd67eab00bfc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:04:58.922158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 864/1750] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:04:58 2022] DELETE /identity/v3/users/4037a903acb945d78ef4cd67eab00bfc => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:04:58.923834 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce6716-1ddb-4bca-b7ad-40398db858d3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:58.924283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce6716-1ddb-4bca-b7ad-40398db858d3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:58.925818 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce6716-1ddb-4bca-b7ad-40398db858d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:58.946259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 887/1751] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:58.946259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404185-c66a-4326-980f-fcc4f4c9a03a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:58.946259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404185-c66a-4326-980f-fcc4f4c9a03a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:58.946259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404185-c66a-4326-980f-fcc4f4c9a03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:58.967426 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ba404185-c66a-4326-980f-fcc4f4c9a03a None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:58.970971 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ba404185-c66a-4326-980f-fcc4f4c9a03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:58.971099 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-50ce6716-1ddb-4bca-b7ad-40398db858d3 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:58.971557 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-50ce6716-1ddb-4bca-b7ad-40398db858d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:59.086974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 865/1752] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:59.086974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd4b3-94d5-4dd5-8152-d20d0daf0990 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.086974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd4b3-94d5-4dd5-8152-d20d0daf0990 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.086974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd4b3-94d5-4dd5-8152-d20d0daf0990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.101819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 888/1753] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:59.106313 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009deda-41c1-4314-9578-fe26ab34086d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.106607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009deda-41c1-4314-9578-fe26ab34086d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.106890 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009deda-41c1-4314-9578-fe26ab34086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.126659 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a80dd4b3-94d5-4dd5-8152-d20d0daf0990 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:59.130143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a80dd4b3-94d5-4dd5-8152-d20d0daf0990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:59.141770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2009deda-41c1-4314-9578-fe26ab34086d None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:04:59.145597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2009deda-41c1-4314-9578-fe26ab34086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:59.154542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 866/1754] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:59.155569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dd4b3-94d5-4dd5-8152-d20d0daf0990 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.157886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a80dd4b3-94d5-4dd5-8152-d20d0daf0990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:59.170472 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 889/1755] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:04:59.171496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009deda-41c1-4314-9578-fe26ab34086d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.183437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009deda-41c1-4314-9578-fe26ab34086d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.194074 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb0846-1b78-4261-896e-7284ecf943a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.195048 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb0846-1b78-4261-896e-7284ecf943a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.195513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb0846-1b78-4261-896e-7284ecf943a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.195968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb0846-1b78-4261-896e-7284ecf943a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.196812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb0846-1b78-4261-896e-7284ecf943a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:59.197319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb0846-1b78-4261-896e-7284ecf943a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.199241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb0846-1b78-4261-896e-7284ecf943a2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.219789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 890/1756] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:59.219789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb0846-1b78-4261-896e-7284ecf943a2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.234282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb0846-1b78-4261-896e-7284ecf943a2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.239894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5b347-b95f-4010-9aad-3fb5f47dc56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.240764 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5b347-b95f-4010-9aad-3fb5f47dc56f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.241592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5b347-b95f-4010-9aad-3fb5f47dc56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.242027 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5b347-b95f-4010-9aad-3fb5f47dc56f admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.243505 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5b347-b95f-4010-9aad-3fb5f47dc56f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:04:59.243971 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5b347-b95f-4010-9aad-3fb5f47dc56f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.245969 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5b347-b95f-4010-9aad-3fb5f47dc56f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.249396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dd4b3-94d5-4dd5-8152-d20d0daf0990 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.258151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356df627-137d-477a-978b-f6ae6094dee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.258151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356df627-137d-477a-978b-f6ae6094dee2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.258151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356df627-137d-477a-978b-f6ae6094dee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.258151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356df627-137d-477a-978b-f6ae6094dee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.273879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356df627-137d-477a-978b-f6ae6094dee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.275110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 891/1757] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:04:59.275597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356df627-137d-477a-978b-f6ae6094dee2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.276495 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5b347-b95f-4010-9aad-3fb5f47dc56f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.288203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 867/1758] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:04:59.289239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356df627-137d-477a-978b-f6ae6094dee2 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.299994 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356df627-137d-477a-978b-f6ae6094dee2 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.304223 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5b347-b95f-4010-9aad-3fb5f47dc56f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.309932 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b0004-d0d3-4b1d-ad52-b9ce50a54357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.311283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b0004-d0d3-4b1d-ad52-b9ce50a54357 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.311777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b0004-d0d3-4b1d-ad52-b9ce50a54357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.312250 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b0004-d0d3-4b1d-ad52-b9ce50a54357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.313110 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b0004-d0d3-4b1d-ad52-b9ce50a54357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.319748 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb76e-da80-49a7-b1b1-00380415b350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.322430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb76e-da80-49a7-b1b1-00380415b350 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.322720 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb76e-da80-49a7-b1b1-00380415b350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.322915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b0004-d0d3-4b1d-ad52-b9ce50a54357 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.323016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb76e-da80-49a7-b1b1-00380415b350 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.323706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb76e-da80-49a7-b1b1-00380415b350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.326362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb76e-da80-49a7-b1b1-00380415b350 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.419018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 892/1759] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:59 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:59.423790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb76e-da80-49a7-b1b1-00380415b350 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.435191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 868/1760] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:04:59 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:04:59.439409 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b0004-d0d3-4b1d-ad52-b9ce50a54357 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.503411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:59.580159 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb76e-da80-49a7-b1b1-00380415b350 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.586525 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6365-e95c-4023-a082-daa68452d52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.587497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6365-e95c-4023-a082-daa68452d52a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.587497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6365-e95c-4023-a082-daa68452d52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.587761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6365-e95c-4023-a082-daa68452d52a admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.588382 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d6365-e95c-4023-a082-daa68452d52a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.590661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d6365-e95c-4023-a082-daa68452d52a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.591111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b0004-d0d3-4b1d-ad52-b9ce50a54357 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.596967 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c710cb-eec1-4451-adc7-377da49c3053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.602577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c710cb-eec1-4451-adc7-377da49c3053 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.602577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c710cb-eec1-4451-adc7-377da49c3053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.602577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c710cb-eec1-4451-adc7-377da49c3053 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.602577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c710cb-eec1-4451-adc7-377da49c3053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.602577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c710cb-eec1-4451-adc7-377da49c3053 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.607713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:04:59.676950 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 893/1761] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:59 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:59.678859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6365-e95c-4023-a082-daa68452d52a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.688234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 869/1762] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:04:59 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:04:59.695395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6365-e95c-4023-a082-daa68452d52a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.699756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c710cb-eec1-4451-adc7-377da49c3053 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.703164 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce5092-13b2-4908-a691-e1d4bee194df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.703962 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce5092-13b2-4908-a691-e1d4bee194df admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.704348 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce5092-13b2-4908-a691-e1d4bee194df admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.704851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce5092-13b2-4908-a691-e1d4bee194df admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.705627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce5092-13b2-4908-a691-e1d4bee194df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.707371 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce5092-13b2-4908-a691-e1d4bee194df admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.714583 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c710cb-eec1-4451-adc7-377da49c3053 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.721087 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddec9de-d5de-4154-8715-215785a06b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.722387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddec9de-d5de-4154-8715-215785a06b6f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.722851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddec9de-d5de-4154-8715-215785a06b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.724633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddec9de-d5de-4154-8715-215785a06b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.724633 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddec9de-d5de-4154-8715-215785a06b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.724822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 894/1763] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:59.727688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddec9de-d5de-4154-8715-215785a06b6f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.730161 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce5092-13b2-4908-a691-e1d4bee194df admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.748116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce5092-13b2-4908-a691-e1d4bee194df admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.750194 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 870/1764] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:59.751265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddec9de-d5de-4154-8715-215785a06b6f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.757349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454338d7-2c79-4001-b091-96a86bebcf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.758097 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454338d7-2c79-4001-b091-96a86bebcf74 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.758411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454338d7-2c79-4001-b091-96a86bebcf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.758719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454338d7-2c79-4001-b091-96a86bebcf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.759411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454338d7-2c79-4001-b091-96a86bebcf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.761422 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454338d7-2c79-4001-b091-96a86bebcf74 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.763901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddec9de-d5de-4154-8715-215785a06b6f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.769343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a65e8-1058-4ab8-8a56-48bc59cccdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.769980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a65e8-1058-4ab8-8a56-48bc59cccdcc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.770337 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a65e8-1058-4ab8-8a56-48bc59cccdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.770710 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a65e8-1058-4ab8-8a56-48bc59cccdcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.771483 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a65e8-1058-4ab8-8a56-48bc59cccdcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.773308 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a65e8-1058-4ab8-8a56-48bc59cccdcc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.777226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 895/1765] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:59.780799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454338d7-2c79-4001-b091-96a86bebcf74 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.785445 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 871/1766] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:04:59.789617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a65e8-1058-4ab8-8a56-48bc59cccdcc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.794096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454338d7-2c79-4001-b091-96a86bebcf74 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.801301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed458590-6674-4ccd-b459-cf3cf47bf9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.802144 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed458590-6674-4ccd-b459-cf3cf47bf9d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.802670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed458590-6674-4ccd-b459-cf3cf47bf9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.802945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a65e8-1058-4ab8-8a56-48bc59cccdcc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:04:59.803074 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed458590-6674-4ccd-b459-cf3cf47bf9d4 admin admin] PATH_INFO: `/v3/projects/013c4952a49b4ce79a514d1efe0d9ee6/users/3c4dae865b094071af6be53ab75f4031/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.808401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c2b68-63ca-43f9-bc2e-0f0a4b219804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:04:59.809145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c2b68-63ca-43f9-bc2e-0f0a4b219804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.810795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c2b68-63ca-43f9-bc2e-0f0a4b219804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.810795 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c2b68-63ca-43f9-bc2e-0f0a4b219804 admin admin] PATH_INFO: `/v3/projects/fe5c02aaba0c41a79346aed612e9a6a2/users/40e9b72b0d224e938e4b9d1539a50600/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.827636 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed458590-6674-4ccd-b459-cf3cf47bf9d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013c4952a49b4ce79a514d1efe0d9ee6, user_id=3c4dae865b094071af6be53ab75f4031, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.831016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c2b68-63ca-43f9-bc2e-0f0a4b219804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5c02aaba0c41a79346aed612e9a6a2, user_id=40e9b72b0d224e938e4b9d1539a50600, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:04:59.834934 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c2b68-63ca-43f9-bc2e-0f0a4b219804 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.839461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed458590-6674-4ccd-b459-cf3cf47bf9d4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:04:59.926109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 872/1767] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:59 2022] PUT /identity/v3/projects/fe5c02aaba0c41a79346aed612e9a6a2/users/40e9b72b0d224e938e4b9d1539a50600/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:59.929180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c2b68-63ca-43f9-bc2e-0f0a4b219804 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:04:59.937335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-248c2b68-63ca-43f9-bc2e-0f0a4b219804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:04:59.949609 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 896/1768] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:04:59 2022] PUT /identity/v3/projects/013c4952a49b4ce79a514d1efe0d9ee6/users/3c4dae865b094071af6be53ab75f4031/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:04:59.951269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d41b-6118-45de-b02e-a15528f0544a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:04:59.951658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d41b-6118-45de-b02e-a15528f0544a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:04:59.952191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d41b-6118-45de-b02e-a15528f0544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:04:59.999235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-75f0d41b-6118-45de-b02e-a15528f0544a None None] MFA Rules not processed for user `40e9b72b0d224e938e4b9d1539a50600`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:00.001245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-75f0d41b-6118-45de-b02e-a15528f0544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:00.072574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c2b68-63ca-43f9-bc2e-0f0a4b219804 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:00.078130 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:00.078790 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:00.079072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:00.079421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] PATH_INFO: `/v3/projects/5a9af80c7f3043e49c3f7da282f77c49` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:00.080565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a9af80c7f3043e49c3f7da282f77c49)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:00.082736 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:00.100103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 897/1769] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:04:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:00.102918 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44dc25-fa03-4779-bd6a-5dc8b1d70453 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:00.103958 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44dc25-fa03-4779-bd6a-5dc8b1d70453 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:00.104448 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44dc25-fa03-4779-bd6a-5dc8b1d70453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:00.144367 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-be44dc25-fa03-4779-bd6a-5dc8b1d70453 None None] MFA Rules not processed for user `3c4dae865b094071af6be53ab75f4031`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:00.144367 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-be44dc25-fa03-4779-bd6a-5dc8b1d70453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:00.220439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 898/1770] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:00.225480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44dc25-fa03-4779-bd6a-5dc8b1d70453 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:00.236730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44dc25-fa03-4779-bd6a-5dc8b1d70453 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:00.241528 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a9af80c7f3043e49c3f7da282f77c49', 'request_id': 'req-cb83bb50-9464-49b3-bd29-faf7d44521a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.249593 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a9af80c7f3043e49c3f7da282f77c49', 'request_id': 'req-cb83bb50-9464-49b3-bd29-faf7d44521a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.269498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:00.298258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55584c-b663-4b50-9a3f-c8052335552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:00.299129 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55584c-b663-4b50-9a3f-c8052335552f admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:00.299564 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55584c-b663-4b50-9a3f-c8052335552f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:00.300375 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55584c-b663-4b50-9a3f-c8052335552f admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:00.301132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55584c-b663-4b50-9a3f-c8052335552f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:00.303408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55584c-b663-4b50-9a3f-c8052335552f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:00.317084 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a9af80c7f3043e49c3f7da282f77c49', 'request_id': 'req-cb83bb50-9464-49b3-bd29-faf7d44521a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.326462 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:00.331861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] The token cache is being invalidate because project 5a9af80c7f3043e49c3f7da282f77c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:00.332158 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.332959 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.334522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 873/1771] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:04:59 2022] DELETE /identity/v3/projects/5a9af80c7f3043e49c3f7da282f77c49 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:00.335323 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:00.336629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:00.404522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83bb50-9464-49b3-bd29-faf7d44521a6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:00.408679 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:00.409028 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:00.409526 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:00.409928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:00.413817 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:00.416779 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 899/1772] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:00 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:00.421344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55584c-b663-4b50-9a3f-c8052335552f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:00.426413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2d55584c-b663-4b50-9a3f-c8052335552f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:00.494276 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:00.495728 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:00.515380 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 874/1773] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:00.516222 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:00.520036 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:00.535446 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55584c-b663-4b50-9a3f-c8052335552f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:00.543167 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c7f46-a103-4546-a853-0a7936cbc9fa service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:00.544925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:00.545811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:00.546179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:00.546553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] PATH_INFO: `/v3/users/9e95006e219e40f3bd2825bc6bf6c95b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:00.547928 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e95006e219e40f3bd2825bc6bf6c95b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:00.549087 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e1e3e-a48d-4957-a69e-cbd936e3d159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:00.550486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1e3e-a48d-4957-a69e-cbd936e3d159 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:00.550843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1e3e-a48d-4957-a69e-cbd936e3d159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:00.550993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:00.551253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1e3e-a48d-4957-a69e-cbd936e3d159 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:00.551939 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e1e3e-a48d-4957-a69e-cbd936e3d159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:00.558512 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e1e3e-a48d-4957-a69e-cbd936e3d159 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:00.682550 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 875/1774] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:00 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:00.684139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e1e3e-a48d-4957-a69e-cbd936e3d159 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:00.686125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6f7e1e3e-a48d-4957-a69e-cbd936e3d159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:00.701446 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e1e3e-a48d-4957-a69e-cbd936e3d159 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:00.710763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:00.711479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:00.711698 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:00.711976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] PATH_INFO: `/v3/users/3c4dae865b094071af6be53ab75f4031` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:00.714016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c4dae865b094071af6be53ab75f4031)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:00.718319 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:00.807410 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e95006e219e40f3bd2825bc6bf6c95b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.821330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e95006e219e40f3bd2825bc6bf6c95b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.822219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e95006e219e40f3bd2825bc6bf6c95b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.823059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e95006e219e40f3bd2825bc6bf6c95b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.887043 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e95006e219e40f3bd2825bc6bf6c95b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.910068 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c4dae865b094071af6be53ab75f4031'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.911480 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c4dae865b094071af6be53ab75f4031'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.929371 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 900/1775] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:00 2022] DELETE /identity/v3/users/9e95006e219e40f3bd2825bc6bf6c95b => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:00.929371 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:00.930074 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c4dae865b094071af6be53ab75f4031'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.931045 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c4dae865b094071af6be53ab75f4031'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:00.931957 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:00.978263 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:01.008191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8a873d93-14d9-4d16-a0d0-ba392063f8cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c4dae865b094071af6be53ab75f4031'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:01.018761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b750-35b2-46a8-8c60-2ef7b206c2dd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:01.026201 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75e84e-52c3-4abf-86d9-31c37e6bcfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:01.027256 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75e84e-52c3-4abf-86d9-31c37e6bcfec admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.027606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75e84e-52c3-4abf-86d9-31c37e6bcfec admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.028039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75e84e-52c3-4abf-86d9-31c37e6bcfec admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.029272 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75e84e-52c3-4abf-86d9-31c37e6bcfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:01.031682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75e84e-52c3-4abf-86d9-31c37e6bcfec admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:01.044600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 876/1776] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:00 2022] DELETE /identity/v3/users/3c4dae865b094071af6be53ab75f4031 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:01.048988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb7dcb-405a-4f32-8a65-ace810c22494 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.049574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb7dcb-405a-4f32-8a65-ace810c22494 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.051331 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 901/1777] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:01.052179 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb7dcb-405a-4f32-8a65-ace810c22494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.053935 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75e84e-52c3-4abf-86d9-31c37e6bcfec admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:01.056589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fe75e84e-52c3-4abf-86d9-31c37e6bcfec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:01.074080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:01.076718 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75e84e-52c3-4abf-86d9-31c37e6bcfec admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:01.080739 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ecdb7dcb-405a-4f32-8a65-ace810c22494 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:01.082103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ecdb7dcb-405a-4f32-8a65-ace810c22494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:01.149661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e2e71-21f6-47e9-aef9-4f5c8a32294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:01.150531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e2e71-21f6-47e9-aef9-4f5c8a32294b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.151085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e2e71-21f6-47e9-aef9-4f5c8a32294b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.151526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e2e71-21f6-47e9-aef9-4f5c8a32294b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.152529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e2e71-21f6-47e9-aef9-4f5c8a32294b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:01.155430 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e2e71-21f6-47e9-aef9-4f5c8a32294b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:01.171314 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 902/1778] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:01.172617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e2e71-21f6-47e9-aef9-4f5c8a32294b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:01.183559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 877/1779] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:01.189535 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e2e71-21f6-47e9-aef9-4f5c8a32294b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:01.200207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb7dcb-405a-4f32-8a65-ace810c22494 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:01.201676 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ecdb7dcb-405a-4f32-8a65-ace810c22494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:01.229722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379e35d-1e5e-4fe4-98f5-6f78b3bf77a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:01.230882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379e35d-1e5e-4fe4-98f5-6f78b3bf77a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.231589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379e35d-1e5e-4fe4-98f5-6f78b3bf77a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.231589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379e35d-1e5e-4fe4-98f5-6f78b3bf77a4 admin admin] PATH_INFO: `/v3/projects/3c760bf4f65943df893ef12e6ab442c8/users/21ce24adb54348db8dfaa1808a109533/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.290986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379e35d-1e5e-4fe4-98f5-6f78b3bf77a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c760bf4f65943df893ef12e6ab442c8, user_id=21ce24adb54348db8dfaa1808a109533, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:01.302707 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379e35d-1e5e-4fe4-98f5-6f78b3bf77a4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:01.331262 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb7dcb-405a-4f32-8a65-ace810c22494 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:01.337899 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7b2fc-473d-4299-8ea6-f082932b4599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:01.341607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7b2fc-473d-4299-8ea6-f082932b4599 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.341607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7b2fc-473d-4299-8ea6-f082932b4599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.341607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7b2fc-473d-4299-8ea6-f082932b4599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.357483 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7b2fc-473d-4299-8ea6-f082932b4599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:01.360151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7b2fc-473d-4299-8ea6-f082932b4599 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:01.380852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 878/1780] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:05:01.391315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 903/1781] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:01 2022] PUT /identity/v3/projects/3c760bf4f65943df893ef12e6ab442c8/users/21ce24adb54348db8dfaa1808a109533/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:01.403587 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c3252-d905-4e4d-b11b-42c7bf53edbc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.403999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c3252-d905-4e4d-b11b-42c7bf53edbc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.404352 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c3252-d905-4e4d-b11b-42c7bf53edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.454097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4e5c3252-d905-4e4d-b11b-42c7bf53edbc None None] MFA Rules not processed for user `21ce24adb54348db8dfaa1808a109533`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:01.455330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4e5c3252-d905-4e4d-b11b-42c7bf53edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:01.548717 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 879/1782] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:01.557357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379e35d-1e5e-4fe4-98f5-6f78b3bf77a4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:01.643330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379e35d-1e5e-4fe4-98f5-6f78b3bf77a4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:01.654645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596e659-deb0-440d-89c9-afe04e766a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:01.654645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596e659-deb0-440d-89c9-afe04e766a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.654645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596e659-deb0-440d-89c9-afe04e766a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.654645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596e659-deb0-440d-89c9-afe04e766a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.655010 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596e659-deb0-440d-89c9-afe04e766a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:01.655010 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596e659-deb0-440d-89c9-afe04e766a92 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:01.735237 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 904/1783] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:01 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:01.740546 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c3252-d905-4e4d-b11b-42c7bf53edbc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:01.752550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a77be9-07cd-4bd7-a4c9-51603edcbef5 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.754044 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a77be9-07cd-4bd7-a4c9-51603edcbef5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.754873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c3252-d905-4e4d-b11b-42c7bf53edbc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:01.755231 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a77be9-07cd-4bd7-a4c9-51603edcbef5 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.757960 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 905/1784] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 13:05:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:05:01.793938 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314ab9a-a38e-40c8-a636-c689f8da0b57 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.794426 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314ab9a-a38e-40c8-a636-c689f8da0b57 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.794705 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314ab9a-a38e-40c8-a636-c689f8da0b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.802767 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:01.820363 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4314ab9a-a38e-40c8-a636-c689f8da0b57 None None] MFA Rules not processed for user `18df1f401bb2494299eda9582c85a04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:01.823079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4314ab9a-a38e-40c8-a636-c689f8da0b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:01.827282 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad4d77-69e5-4bc6-9cc4-3d187078c157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:01.827908 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad4d77-69e5-4bc6-9cc4-3d187078c157 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.828234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad4d77-69e5-4bc6-9cc4-3d187078c157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.828468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad4d77-69e5-4bc6-9cc4-3d187078c157 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.829116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad4d77-69e5-4bc6-9cc4-3d187078c157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:01.831148 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad4d77-69e5-4bc6-9cc4-3d187078c157 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:01.851154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:01.897126 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 906/1785] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 13:05:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:01.898148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314ab9a-a38e-40c8-a636-c689f8da0b57 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:01.899509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4314ab9a-a38e-40c8-a636-c689f8da0b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:01.913315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314ab9a-a38e-40c8-a636-c689f8da0b57 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:01.917980 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:01.918606 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.918890 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.919169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] PATH_INFO: `/v3/projects/013c4952a49b4ce79a514d1efe0d9ee6` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:01.920306 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=013c4952a49b4ce79a514d1efe0d9ee6)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:01.922286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:01.964844 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 880/1786] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:01 2022] POST /identity/v3/users => generated 299 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:01.964844 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad4d77-69e5-4bc6-9cc4-3d187078c157 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:01.964844 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-58ad4d77-69e5-4bc6-9cc4-3d187078c157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:01.984143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad4d77-69e5-4bc6-9cc4-3d187078c157 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:01.991944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12f136-b33e-4ce9-be3a-77b45de591d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18df1f401bb2494299eda9582c85a04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:01.992635 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12f136-b33e-4ce9-be3a-77b45de591d9 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:01.992923 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12f136-b33e-4ce9-be3a-77b45de591d9 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:01.993343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12f136-b33e-4ce9-be3a-77b45de591d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.003750 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12f136-b33e-4ce9-be3a-77b45de591d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.006833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12f136-b33e-4ce9-be3a-77b45de591d9 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.016476 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 881/1787] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:02.017404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12f136-b33e-4ce9-be3a-77b45de591d9 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.018955 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0b12f136-b33e-4ce9-be3a-77b45de591d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.034883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12f136-b33e-4ce9-be3a-77b45de591d9 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.048321 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '013c4952a49b4ce79a514d1efe0d9ee6', 'request_id': 'req-94e348ef-a2cb-437c-9372-85f0574b0b73'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.108437 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.109072 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.109382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.109659 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] PATH_INFO: `/v3/users/83bf124cd05147718d1d7a062fed7c64` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.110798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bf124cd05147718d1d7a062fed7c64)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.113410 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.155465 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '013c4952a49b4ce79a514d1efe0d9ee6', 'request_id': 'req-94e348ef-a2cb-437c-9372-85f0574b0b73'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.163173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '013c4952a49b4ce79a514d1efe0d9ee6', 'request_id': 'req-94e348ef-a2cb-437c-9372-85f0574b0b73'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.174455 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] The token cache is being invalidate because project 013c4952a49b4ce79a514d1efe0d9ee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:02.175419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.176142 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.178047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 907/1788] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:01 2022] DELETE /identity/v3/projects/013c4952a49b4ce79a514d1efe0d9ee6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:02.178943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.180265 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.192921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e348ef-a2cb-437c-9372-85f0574b0b73 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.197603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.198214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.198502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.198773 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] PATH_INFO: `/v3/projects/47ae602122874952aa27755e352634ba` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.199881 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ae602122874952aa27755e352634ba)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.201834 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.265661 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bf124cd05147718d1d7a062fed7c64'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.266331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bf124cd05147718d1d7a062fed7c64'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.274279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bf124cd05147718d1d7a062fed7c64'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.275010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bf124cd05147718d1d7a062fed7c64'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.288951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ae602122874952aa27755e352634ba', 'request_id': 'req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.349451 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bf124cd05147718d1d7a062fed7c64'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.367636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 882/1789] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:02 2022] DELETE /identity/v3/users/83bf124cd05147718d1d7a062fed7c64 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:02.368492 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.369869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.406948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ae602122874952aa27755e352634ba', 'request_id': 'req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.416255 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ae602122874952aa27755e352634ba', 'request_id': 'req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.419896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:02.432409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] The token cache is being invalidate because project 47ae602122874952aa27755e352634ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:02.432706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.433422 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0658b5bd-c04a-4f4b-a13e-93a4d71ec677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.435981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 908/1790] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:02 2022] DELETE /identity/v3/projects/47ae602122874952aa27755e352634ba => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:02.437454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95684f-4062-4ddf-a30e-4abae1560198 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.437734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95684f-4062-4ddf-a30e-4abae1560198 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.438005 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95684f-4062-4ddf-a30e-4abae1560198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.450747 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74f7b1-2e57-4149-9408-8db76dd15823 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.456704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a6e4-34ee-4e97-ab3a-a247235cac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.457430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a6e4-34ee-4e97-ab3a-a247235cac37 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.457791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a6e4-34ee-4e97-ab3a-a247235cac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.458071 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a6e4-34ee-4e97-ab3a-a247235cac37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.458718 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a6e4-34ee-4e97-ab3a-a247235cac37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.460349 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a6e4-34ee-4e97-ab3a-a247235cac37 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.461886 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ff95684f-4062-4ddf-a30e-4abae1560198 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:02.464041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ff95684f-4062-4ddf-a30e-4abae1560198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.474238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 883/1791] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:02.474373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ceaa8-9b08-449d-a020-dd10b1772fbe None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.474579 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ceaa8-9b08-449d-a020-dd10b1772fbe None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.474849 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ceaa8-9b08-449d-a020-dd10b1772fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.478076 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 909/1792] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:02.478991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95684f-4062-4ddf-a30e-4abae1560198 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.480359 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ff95684f-4062-4ddf-a30e-4abae1560198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.495870 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95684f-4062-4ddf-a30e-4abae1560198 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.503091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-779ceaa8-9b08-449d-a020-dd10b1772fbe None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:02.504467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-779ceaa8-9b08-449d-a020-dd10b1772fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.504944 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b10c3-c934-421d-9c90-f8686b4fb2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.506386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b10c3-c934-421d-9c90-f8686b4fb2c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.506679 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b10c3-c934-421d-9c90-f8686b4fb2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.507034 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b10c3-c934-421d-9c90-f8686b4fb2c9 admin admin] PATH_INFO: `/v3/projects/fa934b86c1df4434b1e564404eccfd6f/users/6aef2896ad424858b5cca14044c9142c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.519702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 884/1793] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:02.519702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cac4a3-8b5b-46e9-8bea-f367916ae340 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.519702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cac4a3-8b5b-46e9-8bea-f367916ae340 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.519702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cac4a3-8b5b-46e9-8bea-f367916ae340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.538638 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c5cac4a3-8b5b-46e9-8bea-f367916ae340 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:02.541480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b10c3-c934-421d-9c90-f8686b4fb2c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa934b86c1df4434b1e564404eccfd6f, user_id=6aef2896ad424858b5cca14044c9142c, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.544033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c5cac4a3-8b5b-46e9-8bea-f367916ae340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.546021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b10c3-c934-421d-9c90-f8686b4fb2c9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.556548 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 885/1794] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:02.557925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317c9e6-f96b-4f2b-b4cb-10cebf11e7b7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.558213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317c9e6-f96b-4f2b-b4cb-10cebf11e7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.558491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317c9e6-f96b-4f2b-b4cb-10cebf11e7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.578447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5317c9e6-f96b-4f2b-b4cb-10cebf11e7b7 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:02.579646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5317c9e6-f96b-4f2b-b4cb-10cebf11e7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.590216 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 886/1795] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:02.593504 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317c9e6-f96b-4f2b-b4cb-10cebf11e7b7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.608851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317c9e6-f96b-4f2b-b4cb-10cebf11e7b7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.608851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b6202-62cc-4c6a-87b6-b15212eea9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.608851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b6202-62cc-4c6a-87b6-b15212eea9dc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.608851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b6202-62cc-4c6a-87b6-b15212eea9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.609266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b6202-62cc-4c6a-87b6-b15212eea9dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.609266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b6202-62cc-4c6a-87b6-b15212eea9dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:02.609266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b6202-62cc-4c6a-87b6-b15212eea9dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.610266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b6202-62cc-4c6a-87b6-b15212eea9dc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.620060 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:02.622607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 887/1796] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:05:02.623599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b6202-62cc-4c6a-87b6-b15212eea9dc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.626622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 910/1797] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:02 2022] PUT /identity/v3/projects/fa934b86c1df4434b1e564404eccfd6f/users/6aef2896ad424858b5cca14044c9142c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:02.629582 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b10c3-c934-421d-9c90-f8686b4fb2c9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.635836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b6202-62cc-4c6a-87b6-b15212eea9dc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.725545 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b10c3-c934-421d-9c90-f8686b4fb2c9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.725782 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93d8f3-6d40-4134-86eb-9b1515dfa0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.726262 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93d8f3-6d40-4134-86eb-9b1515dfa0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.726542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93d8f3-6d40-4134-86eb-9b1515dfa0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.726836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93d8f3-6d40-4134-86eb-9b1515dfa0c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.727469 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93d8f3-6d40-4134-86eb-9b1515dfa0c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:02.727779 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93d8f3-6d40-4134-86eb-9b1515dfa0c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.729534 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93d8f3-6d40-4134-86eb-9b1515dfa0c6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.730655 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8a214-dd00-49e3-9a90-2df64c53bb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.731471 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8a214-dd00-49e3-9a90-2df64c53bb99 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.731753 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8a214-dd00-49e3-9a90-2df64c53bb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.732524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8a214-dd00-49e3-9a90-2df64c53bb99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.733158 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8a214-dd00-49e3-9a90-2df64c53bb99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.735168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8a214-dd00-49e3-9a90-2df64c53bb99 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.742023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 888/1798] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:05:02.742873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93d8f3-6d40-4134-86eb-9b1515dfa0c6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.753127 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93d8f3-6d40-4134-86eb-9b1515dfa0c6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.758174 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693a4f4-bd43-4bd7-8a3b-43652cc72f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.758774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693a4f4-bd43-4bd7-8a3b-43652cc72f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.759055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693a4f4-bd43-4bd7-8a3b-43652cc72f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.759334 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693a4f4-bd43-4bd7-8a3b-43652cc72f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.759974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693a4f4-bd43-4bd7-8a3b-43652cc72f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.761624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693a4f4-bd43-4bd7-8a3b-43652cc72f93 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.772270 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 889/1799] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:02.774343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693a4f4-bd43-4bd7-8a3b-43652cc72f93 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.774678 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9693a4f4-bd43-4bd7-8a3b-43652cc72f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.787574 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693a4f4-bd43-4bd7-8a3b-43652cc72f93 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.792175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.792785 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.793066 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.797586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] PATH_INFO: `/v3/projects/1c598340dc584602a53ca1860d932f87` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.797586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c598340dc584602a53ca1860d932f87)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.797586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.797814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 911/1800] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:02 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:02.797814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8a214-dd00-49e3-9a90-2df64c53bb99 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.863074 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8a214-dd00-49e3-9a90-2df64c53bb99 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.882457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c598340dc584602a53ca1860d932f87', 'request_id': 'req-9a0ca091-9c1d-43ac-a968-1a9e78548d60'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.890209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c598340dc584602a53ca1860d932f87', 'request_id': 'req-9a0ca091-9c1d-43ac-a968-1a9e78548d60'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.921459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c6738-8a82-41f7-9fae-7d060c7ccef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.922108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c6738-8a82-41f7-9fae-7d060c7ccef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.922386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c6738-8a82-41f7-9fae-7d060c7ccef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.922658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c6738-8a82-41f7-9fae-7d060c7ccef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.923290 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c6738-8a82-41f7-9fae-7d060c7ccef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.925345 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c6738-8a82-41f7-9fae-7d060c7ccef6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.938613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c598340dc584602a53ca1860d932f87', 'request_id': 'req-9a0ca091-9c1d-43ac-a968-1a9e78548d60'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.950499 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] The token cache is being invalidate because project 1c598340dc584602a53ca1860d932f87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:02.950791 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.951440 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:02.952970 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 890/1801] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:02 2022] DELETE /identity/v3/projects/1c598340dc584602a53ca1860d932f87 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:02.953830 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.955095 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.964509 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:02.968406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ca091-9c1d-43ac-a968-1a9e78548d60 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:02.973419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d295f-3595-4045-805a-fa6bd3604757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:02.974074 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d295f-3595-4045-805a-fa6bd3604757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:02.974388 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d295f-3595-4045-805a-fa6bd3604757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:02.974721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d295f-3595-4045-805a-fa6bd3604757 admin admin] PATH_INFO: `/v3/projects/fa934b86c1df4434b1e564404eccfd6f/users/6aef2896ad424858b5cca14044c9142c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:02.976656 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d295f-3595-4045-805a-fa6bd3604757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa934b86c1df4434b1e564404eccfd6f, user_id=6aef2896ad424858b5cca14044c9142c, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:02.980207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d295f-3595-4045-805a-fa6bd3604757 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:02.990005 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 891/1802] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:02 2022] PUT /identity/v3/projects/fa934b86c1df4434b1e564404eccfd6f/users/6aef2896ad424858b5cca14044c9142c/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:02.990832 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d295f-3595-4045-805a-fa6bd3604757 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:02.992489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d97d295f-3595-4045-805a-fa6bd3604757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:02.997586 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 912/1803] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:02 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:03.001982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72f64-9aa7-4186-9629-6aaf7b12a3b8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.002264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72f64-9aa7-4186-9629-6aaf7b12a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.002545 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72f64-9aa7-4186-9629-6aaf7b12a3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.025163 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-10e72f64-9aa7-4186-9629-6aaf7b12a3b8 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:03.026415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-10e72f64-9aa7-4186-9629-6aaf7b12a3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:03.037469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:03.062169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d295f-3595-4045-805a-fa6bd3604757 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.088610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 913/1804] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:03.089967 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a3fd7-31f9-457d-ad60-752b6dc93538 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.090259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a3fd7-31f9-457d-ad60-752b6dc93538 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.090595 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a3fd7-31f9-457d-ad60-752b6dc93538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.118310 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a35a3fd7-31f9-457d-ad60-752b6dc93538 None None] MFA Rules not processed for user `6aef2896ad424858b5cca14044c9142c`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:03.119615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a35a3fd7-31f9-457d-ad60-752b6dc93538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:03.122452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d88bb1-98da-4bea-89e1-59a0027561ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.123115 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d88bb1-98da-4bea-89e1-59a0027561ef admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.130727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d88bb1-98da-4bea-89e1-59a0027561ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.130727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d88bb1-98da-4bea-89e1-59a0027561ef admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.130727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d88bb1-98da-4bea-89e1-59a0027561ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.130727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d88bb1-98da-4bea-89e1-59a0027561ef admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.190135 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 914/1805] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:03.191610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a3fd7-31f9-457d-ad60-752b6dc93538 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.192891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a35a3fd7-31f9-457d-ad60-752b6dc93538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:03.207107 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a3fd7-31f9-457d-ad60-752b6dc93538 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.212116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ad8b-2012-4559-8713-12550411bb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.216498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ad8b-2012-4559-8713-12550411bb24 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.216498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ad8b-2012-4559-8713-12550411bb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.216498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ad8b-2012-4559-8713-12550411bb24 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.216498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9ad8b-2012-4559-8713-12550411bb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.216498 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9ad8b-2012-4559-8713-12550411bb24 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.241912 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 892/1806] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:02 2022] POST /identity/v3/users => generated 296 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:03.243306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14854084-5042-4d04-bcb2-2227aeb9dd92 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.243702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14854084-5042-4d04-bcb2-2227aeb9dd92 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.244047 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14854084-5042-4d04-bcb2-2227aeb9dd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.263084 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-14854084-5042-4d04-bcb2-2227aeb9dd92 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:03.264279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-14854084-5042-4d04-bcb2-2227aeb9dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:03.272560 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 893/1807] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:03.273484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14854084-5042-4d04-bcb2-2227aeb9dd92 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.274691 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-14854084-5042-4d04-bcb2-2227aeb9dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:03.318572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 915/1808] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:03 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:03.318572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ad8b-2012-4559-8713-12550411bb24 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.326344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ad8b-2012-4559-8713-12550411bb24 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.331712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856df4ec-bb4e-4263-9923-0352de4fe9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.332316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856df4ec-bb4e-4263-9923-0352de4fe9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.332613 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856df4ec-bb4e-4263-9923-0352de4fe9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.332868 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856df4ec-bb4e-4263-9923-0352de4fe9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.333557 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856df4ec-bb4e-4263-9923-0352de4fe9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.335162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856df4ec-bb4e-4263-9923-0352de4fe9f0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.335310 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14854084-5042-4d04-bcb2-2227aeb9dd92 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.339062 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a78cfb-82bf-4e0d-bcfc-d5ebb8dd9b6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.339696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a78cfb-82bf-4e0d-bcfc-d5ebb8dd9b6d service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.339973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a78cfb-82bf-4e0d-bcfc-d5ebb8dd9b6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.340246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a78cfb-82bf-4e0d-bcfc-d5ebb8dd9b6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.342119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-16a78cfb-82bf-4e0d-bcfc-d5ebb8dd9b6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:03.345395 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 916/1809] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:03.346168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856df4ec-bb4e-4263-9923-0352de4fe9f0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.358040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856df4ec-bb4e-4263-9923-0352de4fe9f0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.362960 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02899084-9511-427b-8160-1f276907c6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.363549 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899084-9511-427b-8160-1f276907c6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.363831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899084-9511-427b-8160-1f276907c6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.364103 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899084-9511-427b-8160-1f276907c6d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.364743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02899084-9511-427b-8160-1f276907c6d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:03.365047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02899084-9511-427b-8160-1f276907c6d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.366787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02899084-9511-427b-8160-1f276907c6d8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.379146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 917/1810] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:05:03.379889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02899084-9511-427b-8160-1f276907c6d8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.381954 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:03.390329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02899084-9511-427b-8160-1f276907c6d8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.395779 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254abcac-675c-48b8-a382-ff4e43e84a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.396379 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254abcac-675c-48b8-a382-ff4e43e84a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.396663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254abcac-675c-48b8-a382-ff4e43e84a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.397590 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254abcac-675c-48b8-a382-ff4e43e84a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.397682 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254abcac-675c-48b8-a382-ff4e43e84a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.399316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254abcac-675c-48b8-a382-ff4e43e84a5b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.406189 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a78cfb-82bf-4e0d-bcfc-d5ebb8dd9b6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.407683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a78cfb-82bf-4e0d-bcfc-d5ebb8dd9b6d service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.410368 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 918/1811] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:03.411063 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254abcac-675c-48b8-a382-ff4e43e84a5b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.418326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 894/1812] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:03.419612 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a78cfb-82bf-4e0d-bcfc-d5ebb8dd9b6d service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.420859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254abcac-675c-48b8-a382-ff4e43e84a5b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.425818 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289ce56-b80a-4c8f-84e5-d02cffb8d78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.426424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289ce56-b80a-4c8f-84e5-d02cffb8d78b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.432211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289ce56-b80a-4c8f-84e5-d02cffb8d78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.432211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289ce56-b80a-4c8f-84e5-d02cffb8d78b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.432211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289ce56-b80a-4c8f-84e5-d02cffb8d78b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.432211 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289ce56-b80a-4c8f-84e5-d02cffb8d78b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.432501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a78cfb-82bf-4e0d-bcfc-d5ebb8dd9b6d service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.437173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16d757-092f-4091-aa25-29b1ee1e4b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.438104 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16d757-092f-4091-aa25-29b1ee1e4b01 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.438393 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16d757-092f-4091-aa25-29b1ee1e4b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.438672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16d757-092f-4091-aa25-29b1ee1e4b01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.439304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16d757-092f-4091-aa25-29b1ee1e4b01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.441017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 919/1813] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:03.441669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289ce56-b80a-4c8f-84e5-d02cffb8d78b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.444768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16d757-092f-4091-aa25-29b1ee1e4b01 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.451679 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289ce56-b80a-4c8f-84e5-d02cffb8d78b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.457091 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53491ffd-f866-4f4b-8e1b-a0738226ba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.457745 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53491ffd-f866-4f4b-8e1b-a0738226ba4d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.458154 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53491ffd-f866-4f4b-8e1b-a0738226ba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.458444 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53491ffd-f866-4f4b-8e1b-a0738226ba4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.459091 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53491ffd-f866-4f4b-8e1b-a0738226ba4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.460977 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53491ffd-f866-4f4b-8e1b-a0738226ba4d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.472749 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 920/1814] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:03.473619 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53491ffd-f866-4f4b-8e1b-a0738226ba4d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.483796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53491ffd-f866-4f4b-8e1b-a0738226ba4d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.488860 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d027294-cb7b-424b-a944-4dbf792acc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.489553 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d027294-cb7b-424b-a944-4dbf792acc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.489830 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d027294-cb7b-424b-a944-4dbf792acc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.490104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d027294-cb7b-424b-a944-4dbf792acc0b admin admin] PATH_INFO: `/v3/projects/b0923e472dbe4ae18809c2748a4ff9a1/users/c45853aaa3374114800a5f2957785126/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.505232 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d027294-cb7b-424b-a944-4dbf792acc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0923e472dbe4ae18809c2748a4ff9a1, user_id=c45853aaa3374114800a5f2957785126, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.507901 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d027294-cb7b-424b-a944-4dbf792acc0b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.573858 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 895/1815] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:03 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:03.574733 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16d757-092f-4091-aa25-29b1ee1e4b01 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.602449 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 921/1816] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:03 2022] PUT /identity/v3/projects/b0923e472dbe4ae18809c2748a4ff9a1/users/c45853aaa3374114800a5f2957785126/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:03.603490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d027294-cb7b-424b-a944-4dbf792acc0b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.615266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:03.644125 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16d757-092f-4091-aa25-29b1ee1e4b01 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.674838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d027294-cb7b-424b-a944-4dbf792acc0b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.679999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec39a1-6b17-4001-9882-301893d18802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.680604 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec39a1-6b17-4001-9882-301893d18802 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.680912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec39a1-6b17-4001-9882-301893d18802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.681182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec39a1-6b17-4001-9882-301893d18802 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.681842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec39a1-6b17-4001-9882-301893d18802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.683767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec39a1-6b17-4001-9882-301893d18802 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.699991 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff47690-e0f8-450a-a49b-8ec43d48b33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.700659 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff47690-e0f8-450a-a49b-8ec43d48b33f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.700943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff47690-e0f8-450a-a49b-8ec43d48b33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.701252 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff47690-e0f8-450a-a49b-8ec43d48b33f admin admin] PATH_INFO: `/v3/projects/cd76ac4ece6e4563b372000ea9cb7df9/users/e1c52abee56244b49e02212ac13e0ed6/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.716362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff47690-e0f8-450a-a49b-8ec43d48b33f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd76ac4ece6e4563b372000ea9cb7df9, user_id=e1c52abee56244b49e02212ac13e0ed6, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.719139 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff47690-e0f8-450a-a49b-8ec43d48b33f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.784864 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 922/1817] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:03 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:03.786204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0b361-4579-4c25-8dc5-ac7aeb9da9b6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.786490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0b361-4579-4c25-8dc5-ac7aeb9da9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.786786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0b361-4579-4c25-8dc5-ac7aeb9da9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.809201 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 896/1818] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:03 2022] PUT /identity/v3/projects/cd76ac4ece6e4563b372000ea9cb7df9/users/e1c52abee56244b49e02212ac13e0ed6/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:03.810067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff47690-e0f8-450a-a49b-8ec43d48b33f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.821849 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2cb0b361-4579-4c25-8dc5-ac7aeb9da9b6 None None] MFA Rules not processed for user `c45853aaa3374114800a5f2957785126`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:03.823409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2cb0b361-4579-4c25-8dc5-ac7aeb9da9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:03.863880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:03.899672 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff47690-e0f8-450a-a49b-8ec43d48b33f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.904524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0f970-ec58-4ec2-aaee-65641811e62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.905140 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0f970-ec58-4ec2-aaee-65641811e62e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.905454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0f970-ec58-4ec2-aaee-65641811e62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.905734 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0f970-ec58-4ec2-aaee-65641811e62e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.906381 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0f970-ec58-4ec2-aaee-65641811e62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.908010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0f970-ec58-4ec2-aaee-65641811e62e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.913588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 923/1819] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:03.914975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce05185-89fc-446d-9954-406e9a3ccbe1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.915264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce05185-89fc-446d-9954-406e9a3ccbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.915539 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce05185-89fc-446d-9954-406e9a3ccbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.919562 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 897/1820] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:03.922826 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0f970-ec58-4ec2-aaee-65641811e62e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.933592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0f970-ec58-4ec2-aaee-65641811e62e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:03.938609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78457b-435f-4cd1-a717-2c7ad7a7df8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:03.939294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78457b-435f-4cd1-a717-2c7ad7a7df8a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:03.939576 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78457b-435f-4cd1-a717-2c7ad7a7df8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:03.939852 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78457b-435f-4cd1-a717-2c7ad7a7df8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:03.941266 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78457b-435f-4cd1-a717-2c7ad7a7df8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:03.942988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78457b-435f-4cd1-a717-2c7ad7a7df8a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:03.944975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1ce05185-89fc-446d-9954-406e9a3ccbe1 None None] MFA Rules not processed for user `e1c52abee56244b49e02212ac13e0ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:03.946307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1ce05185-89fc-446d-9954-406e9a3ccbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:03.953182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 898/1821] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:05:03.954009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78457b-435f-4cd1-a717-2c7ad7a7df8a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:03.955630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7c78457b-435f-4cd1-a717-2c7ad7a7df8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:04.014856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 924/1822] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:04.015688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce05185-89fc-446d-9954-406e9a3ccbe1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.016686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78457b-435f-4cd1-a717-2c7ad7a7df8a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.021250 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baae7b5-5def-4da8-8e1a-7f3cd35b5f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.021874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baae7b5-5def-4da8-8e1a-7f3cd35b5f26 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.023360 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baae7b5-5def-4da8-8e1a-7f3cd35b5f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.023360 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baae7b5-5def-4da8-8e1a-7f3cd35b5f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.028047 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce05185-89fc-446d-9954-406e9a3ccbe1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.031062 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baae7b5-5def-4da8-8e1a-7f3cd35b5f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.032497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baae7b5-5def-4da8-8e1a-7f3cd35b5f26 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.033298 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3c12e-0347-4eb0-abd7-18c749e6bd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.034050 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3c12e-0347-4eb0-abd7-18c749e6bd57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.034482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3c12e-0347-4eb0-abd7-18c749e6bd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.034851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3c12e-0347-4eb0-abd7-18c749e6bd57 admin admin] PATH_INFO: `/v3/projects/36ac569f72814a7aac786d91843cb362/users/b80b889cad234796943431cd6b3e80a8/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.041522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 899/1823] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:04.042301 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baae7b5-5def-4da8-8e1a-7f3cd35b5f26 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.045700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:04.057629 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3c12e-0347-4eb0-abd7-18c749e6bd57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ac569f72814a7aac786d91843cb362, user_id=b80b889cad234796943431cd6b3e80a8, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.057757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baae7b5-5def-4da8-8e1a-7f3cd35b5f26 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.057855 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3c12e-0347-4eb0-abd7-18c749e6bd57 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.059670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483cad8-d364-40e5-96d5-6d68dab52eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.060675 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483cad8-d364-40e5-96d5-6d68dab52eca admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.060972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483cad8-d364-40e5-96d5-6d68dab52eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.061247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483cad8-d364-40e5-96d5-6d68dab52eca admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.061883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483cad8-d364-40e5-96d5-6d68dab52eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.064425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483cad8-d364-40e5-96d5-6d68dab52eca admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.178986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 925/1824] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:04 2022] PUT /identity/v3/projects/36ac569f72814a7aac786d91843cb362/users/b80b889cad234796943431cd6b3e80a8/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:04.189068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c34257-3895-4677-afc2-248d5d9ba506 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.189407 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c34257-3895-4677-afc2-248d5d9ba506 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.189709 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c34257-3895-4677-afc2-248d5d9ba506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.210731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 900/1825] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:04 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:04.214680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483cad8-d364-40e5-96d5-6d68dab52eca admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.222791 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d3c34257-3895-4677-afc2-248d5d9ba506 None None] MFA Rules not processed for user `b80b889cad234796943431cd6b3e80a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:04.224092 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d3c34257-3895-4677-afc2-248d5d9ba506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:04.290663 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483cad8-d364-40e5-96d5-6d68dab52eca admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.292677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 926/1826] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:04.297078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c34257-3895-4677-afc2-248d5d9ba506 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.299115 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9ce23-02de-4a9a-88bb-16a6057165c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.299729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9ce23-02de-4a9a-88bb-16a6057165c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.300015 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9ce23-02de-4a9a-88bb-16a6057165c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.300326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9ce23-02de-4a9a-88bb-16a6057165c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.300969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9ce23-02de-4a9a-88bb-16a6057165c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.303226 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9ce23-02de-4a9a-88bb-16a6057165c9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.307715 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c34257-3895-4677-afc2-248d5d9ba506 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.312423 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488876a-238f-4528-bd63-f93b199f1749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.313021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488876a-238f-4528-bd63-f93b199f1749 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.313343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488876a-238f-4528-bd63-f93b199f1749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.313623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488876a-238f-4528-bd63-f93b199f1749 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.314245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488876a-238f-4528-bd63-f93b199f1749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.316204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488876a-238f-4528-bd63-f93b199f1749 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.386293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 901/1827] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:04 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:05:04.390088 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9ce23-02de-4a9a-88bb-16a6057165c9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.402463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9ce23-02de-4a9a-88bb-16a6057165c9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.408041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c415d-5cc4-46f1-90e8-884a021419c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.408642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c415d-5cc4-46f1-90e8-884a021419c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.408982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c415d-5cc4-46f1-90e8-884a021419c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.409811 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c415d-5cc4-46f1-90e8-884a021419c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.410465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c415d-5cc4-46f1-90e8-884a021419c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.412230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c415d-5cc4-46f1-90e8-884a021419c6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.423508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 902/1828] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:05:04.428141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c415d-5cc4-46f1-90e8-884a021419c6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.439411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c415d-5cc4-46f1-90e8-884a021419c6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.442874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbcd16-0a14-48bd-9d6b-3b68df1a0870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.443510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbcd16-0a14-48bd-9d6b-3b68df1a0870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.443789 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbcd16-0a14-48bd-9d6b-3b68df1a0870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.444067 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbcd16-0a14-48bd-9d6b-3b68df1a0870 admin admin] PATH_INFO: `/v3/projects/afb0b54f68e743afa9eca36c28e3e989/users/88020d4d8f5741aa802570ceae811827/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.459673 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbcd16-0a14-48bd-9d6b-3b68df1a0870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb0b54f68e743afa9eca36c28e3e989, user_id=88020d4d8f5741aa802570ceae811827, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.462400 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbcd16-0a14-48bd-9d6b-3b68df1a0870 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.471246 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 927/1829] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:04 2022] POST /identity/v3/projects => generated 363 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:05:04.486909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488876a-238f-4528-bd63-f93b199f1749 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.545897 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488876a-238f-4528-bd63-f93b199f1749 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.551919 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b896cb-c41b-4d23-b328-385a678cf511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.552941 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b896cb-c41b-4d23-b328-385a678cf511 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.553274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b896cb-c41b-4d23-b328-385a678cf511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.553537 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b896cb-c41b-4d23-b328-385a678cf511 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.554467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b896cb-c41b-4d23-b328-385a678cf511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.556568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b896cb-c41b-4d23-b328-385a678cf511 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.641255 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 903/1830] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:04 2022] PUT /identity/v3/projects/afb0b54f68e743afa9eca36c28e3e989/users/88020d4d8f5741aa802570ceae811827/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:04.645926 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbcd16-0a14-48bd-9d6b-3b68df1a0870 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.651277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:04.672240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 928/1831] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:04 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:05:04.675848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b896cb-c41b-4d23-b328-385a678cf511 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.687127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b896cb-c41b-4d23-b328-385a678cf511 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.716452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbcd16-0a14-48bd-9d6b-3b68df1a0870 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.722461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0981716-ebd3-4a24-9d2a-f2148b6a42b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.723082 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0981716-ebd3-4a24-9d2a-f2148b6a42b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.723366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0981716-ebd3-4a24-9d2a-f2148b6a42b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.723644 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0981716-ebd3-4a24-9d2a-f2148b6a42b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.724291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0981716-ebd3-4a24-9d2a-f2148b6a42b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.725960 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0981716-ebd3-4a24-9d2a-f2148b6a42b0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.730638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:04.740615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 904/1832] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:05:04.742629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0981716-ebd3-4a24-9d2a-f2148b6a42b0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.748586 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4f936-0108-480c-996b-ce955e6765b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.749218 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4f936-0108-480c-996b-ce955e6765b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.752538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4f936-0108-480c-996b-ce955e6765b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.752538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4f936-0108-480c-996b-ce955e6765b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.752538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4f936-0108-480c-996b-ce955e6765b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.752538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4f936-0108-480c-996b-ce955e6765b4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.755885 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0981716-ebd3-4a24-9d2a-f2148b6a42b0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.760925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed5090-9784-41f1-a173-5aa892d80d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.761749 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5090-9784-41f1-a173-5aa892d80d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.762039 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5090-9784-41f1-a173-5aa892d80d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.762421 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5090-9784-41f1-a173-5aa892d80d63 admin admin] PATH_INFO: `/v3/projects/afb0b54f68e743afa9eca36c28e3e989/users/88020d4d8f5741aa802570ceae811827/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.767991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 929/1833] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:05:04.768143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed5090-9784-41f1-a173-5aa892d80d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb0b54f68e743afa9eca36c28e3e989, user_id=88020d4d8f5741aa802570ceae811827, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.768143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed5090-9784-41f1-a173-5aa892d80d63 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.768387 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4f936-0108-480c-996b-ce955e6765b4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.778924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4f936-0108-480c-996b-ce955e6765b4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:04.779798 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 905/1834] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:04 2022] PUT /identity/v3/projects/afb0b54f68e743afa9eca36c28e3e989/users/88020d4d8f5741aa802570ceae811827/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:04.790086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57065a-8c7a-46c9-b570-9399f2c366fb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.790484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57065a-8c7a-46c9-b570-9399f2c366fb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.790657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57065a-8c7a-46c9-b570-9399f2c366fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.822580 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4d57065a-8c7a-46c9-b570-9399f2c366fb None None] MFA Rules not processed for user `88020d4d8f5741aa802570ceae811827`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:04.823806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4d57065a-8c7a-46c9-b570-9399f2c366fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:04.846923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b3f8c-06b6-4dcc-b94a-14eb189e4e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:04.850109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b3f8c-06b6-4dcc-b94a-14eb189e4e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:04.850109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b3f8c-06b6-4dcc-b94a-14eb189e4e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:04.850109 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b3f8c-06b6-4dcc-b94a-14eb189e4e6d admin admin] PATH_INFO: `/v3/projects/9439a2c8c69e4c3db29d78f1e61450fa/users/3d73ebd47b4541cca800d71823d179c4/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:04.862954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b3f8c-06b6-4dcc-b94a-14eb189e4e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9439a2c8c69e4c3db29d78f1e61450fa, user_id=3d73ebd47b4541cca800d71823d179c4, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:04.865670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b3f8c-06b6-4dcc-b94a-14eb189e4e6d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:04.894195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 906/1835] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:04.905621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57065a-8c7a-46c9-b570-9399f2c366fb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.948871 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 930/1836] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:04 2022] PUT /identity/v3/projects/9439a2c8c69e4c3db29d78f1e61450fa/users/3d73ebd47b4541cca800d71823d179c4/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:04.950981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b3f8c-06b6-4dcc-b94a-14eb189e4e6d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:04.970727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57065a-8c7a-46c9-b570-9399f2c366fb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:05.006096 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:05.028767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b3f8c-06b6-4dcc-b94a-14eb189e4e6d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:05.034657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc1df0-a417-4f1a-8254-6b640de321a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:05.035467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc1df0-a417-4f1a-8254-6b640de321a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:05.035758 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc1df0-a417-4f1a-8254-6b640de321a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:05.036170 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc1df0-a417-4f1a-8254-6b640de321a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:05.036950 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc1df0-a417-4f1a-8254-6b640de321a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:05.038519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3555e-f459-4fc5-8919-1cfc565efa97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:05.038883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc1df0-a417-4f1a-8254-6b640de321a5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:05.039145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3555e-f459-4fc5-8919-1cfc565efa97 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:05.039486 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3555e-f459-4fc5-8919-1cfc565efa97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:05.039768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3555e-f459-4fc5-8919-1cfc565efa97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:05.052046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 931/1837] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:05.057128 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc1df0-a417-4f1a-8254-6b640de321a5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:05.067297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc1df0-a417-4f1a-8254-6b640de321a5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:05.075127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e49579-383d-4d31-b01e-0034e6a8117e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:05.076312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e49579-383d-4d31-b01e-0034e6a8117e admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:05.077011 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e49579-383d-4d31-b01e-0034e6a8117e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:05.077329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e49579-383d-4d31-b01e-0034e6a8117e admin admin] PATH_INFO: `/v3/projects/9439a2c8c69e4c3db29d78f1e61450fa/users/3d73ebd47b4541cca800d71823d179c4/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:05.079302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e49579-383d-4d31-b01e-0034e6a8117e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9439a2c8c69e4c3db29d78f1e61450fa, user_id=3d73ebd47b4541cca800d71823d179c4, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:05.083396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e49579-383d-4d31-b01e-0034e6a8117e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:05.100981 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 932/1838] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:05 2022] PUT /identity/v3/projects/9439a2c8c69e4c3db29d78f1e61450fa/users/3d73ebd47b4541cca800d71823d179c4/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:05.112460 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afddfe6-b0d7-41a1-9007-e9d7a55ea301 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:05.112776 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afddfe6-b0d7-41a1-9007-e9d7a55ea301 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:05.113147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afddfe6-b0d7-41a1-9007-e9d7a55ea301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:05.115485 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3555e-f459-4fc5-8919-1cfc565efa97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:05.116950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3555e-f459-4fc5-8919-1cfc565efa97 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:05.126726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 907/1839] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:05:05.142700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0afddfe6-b0d7-41a1-9007-e9d7a55ea301 None None] MFA Rules not processed for user `3d73ebd47b4541cca800d71823d179c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:05.144611 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0afddfe6-b0d7-41a1-9007-e9d7a55ea301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:05.179035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:05.215013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 933/1840] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:05.226389 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3555e-f459-4fc5-8919-1cfc565efa97 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:05.235133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3555e-f459-4fc5-8919-1cfc565efa97 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:05.289786 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0add2-11fc-435b-a9f3-d78fc479bb41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:05.290406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0add2-11fc-435b-a9f3-d78fc479bb41 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:05.290686 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0add2-11fc-435b-a9f3-d78fc479bb41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:05.290981 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0add2-11fc-435b-a9f3-d78fc479bb41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:05.300491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0add2-11fc-435b-a9f3-d78fc479bb41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:05.302026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0add2-11fc-435b-a9f3-d78fc479bb41 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:05.314468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 908/1841] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:05.488195 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afddfe6-b0d7-41a1-9007-e9d7a55ea301 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:05.498720 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afddfe6-b0d7-41a1-9007-e9d7a55ea301 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:05.504199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a180ebe-3aff-439c-a5c7-b865d3071a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:05.504867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180ebe-3aff-439c-a5c7-b865d3071a93 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:05.505147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180ebe-3aff-439c-a5c7-b865d3071a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:05.505447 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180ebe-3aff-439c-a5c7-b865d3071a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:05.568892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a180ebe-3aff-439c-a5c7-b865d3071a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:05.570465 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a180ebe-3aff-439c-a5c7-b865d3071a93 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:05.580024 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 934/1842] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:05.670598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0add2-11fc-435b-a9f3-d78fc479bb41 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:05.732592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0add2-11fc-435b-a9f3-d78fc479bb41 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:05.737053 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:05.737847 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:05.738121 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:05.738396 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] PATH_INFO: `/v3/users/e1c52abee56244b49e02212ac13e0ed6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:05.739504 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1c52abee56244b49e02212ac13e0ed6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:05.741448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:05.764448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:05.923705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1c52abee56244b49e02212ac13e0ed6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:05.924384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1c52abee56244b49e02212ac13e0ed6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:05.931843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1c52abee56244b49e02212ac13e0ed6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:05.932482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1c52abee56244b49e02212ac13e0ed6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:06.018186 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1c52abee56244b49e02212ac13e0ed6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:06.037524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 909/1843] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:05 2022] DELETE /identity/v3/users/e1c52abee56244b49e02212ac13e0ed6 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:06.042067 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3792f-191c-4583-bbad-d3462490fe4c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:06.042542 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3792f-191c-4583-bbad-d3462490fe4c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:06.042693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3792f-191c-4583-bbad-d3462490fe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:06.062999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ffe3792f-191c-4583-bbad-d3462490fe4c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:06.064209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ffe3792f-191c-4583-bbad-d3462490fe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:06.129763 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 935/1844] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:06.139693 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:06.141361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:06.203861 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebf467-1599-4da6-93a8-a54c34158498 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:06.209820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafeb66-0310-48bf-8c51-2af9f612fc4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:06.210460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafeb66-0310-48bf-8c51-2af9f612fc4f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:06.210731 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafeb66-0310-48bf-8c51-2af9f612fc4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:06.211009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafeb66-0310-48bf-8c51-2af9f612fc4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:06.222490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafeb66-0310-48bf-8c51-2af9f612fc4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:06.222490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafeb66-0310-48bf-8c51-2af9f612fc4f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:06.232100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 910/1845] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:06.523946 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3792f-191c-4583-bbad-d3462490fe4c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:06.525393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ffe3792f-191c-4583-bbad-d3462490fe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:06.587604 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3792f-191c-4583-bbad-d3462490fe4c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:06.591275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb63a3-fb38-42aa-8fa2-3ac07e8f025c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:06.591883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb63a3-fb38-42aa-8fa2-3ac07e8f025c service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:06.592166 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb63a3-fb38-42aa-8fa2-3ac07e8f025c service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:06.592439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb63a3-fb38-42aa-8fa2-3ac07e8f025c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:06.594214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-13cb63a3-fb38-42aa-8fa2-3ac07e8f025c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:06.620758 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:06.653341 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb63a3-fb38-42aa-8fa2-3ac07e8f025c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:06.654842 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb63a3-fb38-42aa-8fa2-3ac07e8f025c service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:06.660382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafeb66-0310-48bf-8c51-2af9f612fc4f service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:06.661955 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bfafeb66-0310-48bf-8c51-2af9f612fc4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:06.679599 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafeb66-0310-48bf-8c51-2af9f612fc4f service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:06.680030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 936/1846] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:05:06.683163 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:06.684032 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:06.684366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:06.684713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] PATH_INFO: `/v3/projects/cd76ac4ece6e4563b372000ea9cb7df9` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:06.685941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd76ac4ece6e4563b372000ea9cb7df9)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:06.687874 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:06.783542 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd76ac4ece6e4563b372000ea9cb7df9', 'request_id': 'req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:06.790512 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd76ac4ece6e4563b372000ea9cb7df9', 'request_id': 'req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:06.864328 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd76ac4ece6e4563b372000ea9cb7df9', 'request_id': 'req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:06.875444 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] The token cache is being invalidate because project cd76ac4ece6e4563b372000ea9cb7df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:06.875738 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:06.876482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:06.878170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 911/1847] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:06 2022] DELETE /identity/v3/projects/cd76ac4ece6e4563b372000ea9cb7df9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:06.882479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb63a3-fb38-42aa-8fa2-3ac07e8f025c service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:06.894759 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb63a3-fb38-42aa-8fa2-3ac07e8f025c service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:06.976943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:06.976943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:06.976943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:06.976943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] PATH_INFO: `/v3/users/88020d4d8f5741aa802570ceae811827` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:06.977774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88020d4d8f5741aa802570ceae811827)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:06.977774 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:07.189055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88020d4d8f5741aa802570ceae811827'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:07.196346 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88020d4d8f5741aa802570ceae811827'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:07.197030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88020d4d8f5741aa802570ceae811827'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:07.197683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88020d4d8f5741aa802570ceae811827'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:07.246427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b054747-16d6-4f3b-9a48-45876f9307b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88020d4d8f5741aa802570ceae811827'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:07.264543 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 937/1848] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:06 2022] DELETE /identity/v3/users/88020d4d8f5741aa802570ceae811827 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:07.724411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:07.726095 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:07.739446 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50c646-5a80-4a46-99b4-ab1cbf5e61f2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:07.765997 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:07.800083 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:07.800727 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:07.801010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:07.801318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] PATH_INFO: `/v3/projects/afb0b54f68e743afa9eca36c28e3e989` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:07.802707 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb0b54f68e743afa9eca36c28e3e989)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:07.804618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:07.928896 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb0b54f68e743afa9eca36c28e3e989', 'request_id': 'req-c9a86cf7-2afa-4f33-8b03-167635b1ec15'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:07.935554 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb0b54f68e743afa9eca36c28e3e989', 'request_id': 'req-c9a86cf7-2afa-4f33-8b03-167635b1ec15'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:08.041552 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb0b54f68e743afa9eca36c28e3e989', 'request_id': 'req-c9a86cf7-2afa-4f33-8b03-167635b1ec15'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:08.075516 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] The token cache is being invalidate because project afb0b54f68e743afa9eca36c28e3e989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:08.076016 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:08.077230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c9a86cf7-2afa-4f33-8b03-167635b1ec15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:08.080363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 912/1849] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:07 2022] DELETE /identity/v3/projects/afb0b54f68e743afa9eca36c28e3e989 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:08.090824 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39672f-9d7b-4220-91b5-713108c473c4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.091114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39672f-9d7b-4220-91b5-713108c473c4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.095364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39672f-9d7b-4220-91b5-713108c473c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.112513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0e39672f-9d7b-4220-91b5-713108c473c4 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:08.114349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0e39672f-9d7b-4220-91b5-713108c473c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:08.164362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:08.197458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 938/1850] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:08.199990 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbc615-cbe3-4bf4-b5d3-0d3c1f283dd2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.199990 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbc615-cbe3-4bf4-b5d3-0d3c1f283dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.200212 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbc615-cbe3-4bf4-b5d3-0d3c1f283dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.229092 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d4dbc615-cbe3-4bf4-b5d3-0d3c1f283dd2 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:08.230455 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d4dbc615-cbe3-4bf4-b5d3-0d3c1f283dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:08.239640 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39672f-9d7b-4220-91b5-713108c473c4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:08.241253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 913/1851] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:08.241628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0e39672f-9d7b-4220-91b5-713108c473c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:08.244638 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbc615-cbe3-4bf4-b5d3-0d3c1f283dd2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:08.257050 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbc615-cbe3-4bf4-b5d3-0d3c1f283dd2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:08.263385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b5516-0495-4d55-8648-1ab7b9a7c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:08.264053 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b5516-0495-4d55-8648-1ab7b9a7c92e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.264340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b5516-0495-4d55-8648-1ab7b9a7c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.264717 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b5516-0495-4d55-8648-1ab7b9a7c92e admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.270128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b5516-0495-4d55-8648-1ab7b9a7c92e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:08.270128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b5516-0495-4d55-8648-1ab7b9a7c92e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:08.270128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b5516-0495-4d55-8648-1ab7b9a7c92e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:08.285249 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 914/1852] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:05:08.290124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b5516-0495-4d55-8648-1ab7b9a7c92e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:08.297522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b5516-0495-4d55-8648-1ab7b9a7c92e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:08.302570 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368d441-498b-482d-b83a-85cf4b61c7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:08.303170 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368d441-498b-482d-b83a-85cf4b61c7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.303450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368d441-498b-482d-b83a-85cf4b61c7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.303592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39672f-9d7b-4220-91b5-713108c473c4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:08.303703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368d441-498b-482d-b83a-85cf4b61c7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.304338 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368d441-498b-482d-b83a-85cf4b61c7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:08.306293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368d441-498b-482d-b83a-85cf4b61c7b6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:08.307230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fb3ae-0671-4bcd-8ba2-deaccfaaedbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:08.307948 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fb3ae-0671-4bcd-8ba2-deaccfaaedbf service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.308236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fb3ae-0671-4bcd-8ba2-deaccfaaedbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.308509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fb3ae-0671-4bcd-8ba2-deaccfaaedbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.310297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2a7fb3ae-0671-4bcd-8ba2-deaccfaaedbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:08.375910 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fb3ae-0671-4bcd-8ba2-deaccfaaedbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:08.377428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fb3ae-0671-4bcd-8ba2-deaccfaaedbf service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:08.387769 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 939/1853] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:08.406198 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 915/1854] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:08 2022] POST /identity/v3/projects => generated 359 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:08.407352 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fb3ae-0671-4bcd-8ba2-deaccfaaedbf service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:08.418405 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fb3ae-0671-4bcd-8ba2-deaccfaaedbf service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:08.475121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6e7c9-5878-4706-89f6-e869ef530cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:08.475661 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6e7c9-5878-4706-89f6-e869ef530cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.475943 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6e7c9-5878-4706-89f6-e869ef530cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.476219 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6e7c9-5878-4706-89f6-e869ef530cd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.476872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6e7c9-5878-4706-89f6-e869ef530cd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:08.478851 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6e7c9-5878-4706-89f6-e869ef530cd8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:08.576517 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 940/1855] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:08 2022] POST /identity/v3/users => generated 280 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:08.578169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368d441-498b-482d-b83a-85cf4b61c7b6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:08.592232 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368d441-498b-482d-b83a-85cf4b61c7b6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:08.597610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faed350-60da-4469-a236-03b67c6c4b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:08.598234 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed350-60da-4469-a236-03b67c6c4b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.598513 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed350-60da-4469-a236-03b67c6c4b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.598785 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed350-60da-4469-a236-03b67c6c4b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.599995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faed350-60da-4469-a236-03b67c6c4b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:08.601150 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faed350-60da-4469-a236-03b67c6c4b63 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:08.616480 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 916/1856] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:08.617059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6e7c9-5878-4706-89f6-e869ef530cd8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:08.628387 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6e7c9-5878-4706-89f6-e869ef530cd8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:08.633795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e45548-4a51-4ab6-a0d8-b788a0eee67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:08.634415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e45548-4a51-4ab6-a0d8-b788a0eee67a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.634702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e45548-4a51-4ab6-a0d8-b788a0eee67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.639184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e45548-4a51-4ab6-a0d8-b788a0eee67a admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.639184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e45548-4a51-4ab6-a0d8-b788a0eee67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:08.639184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e45548-4a51-4ab6-a0d8-b788a0eee67a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:08.653720 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 941/1857] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:08.657845 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faed350-60da-4469-a236-03b67c6c4b63 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:08.667642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faed350-60da-4469-a236-03b67c6c4b63 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:08.672385 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215b09b-f63b-4eb0-bb0a-cb3de72a7ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:08.673042 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215b09b-f63b-4eb0-bb0a-cb3de72a7ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.673355 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215b09b-f63b-4eb0-bb0a-cb3de72a7ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.673642 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215b09b-f63b-4eb0-bb0a-cb3de72a7ed7 admin admin] PATH_INFO: `/v3/projects/67baab2427ad4223b3399d193b14bc09/users/449df1d62a9d4a6da2747c223dced904/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.689758 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215b09b-f63b-4eb0-bb0a-cb3de72a7ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67baab2427ad4223b3399d193b14bc09, user_id=449df1d62a9d4a6da2747c223dced904, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:08.693986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215b09b-f63b-4eb0-bb0a-cb3de72a7ed7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:08.790009 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 917/1858] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:08 2022] PUT /identity/v3/projects/67baab2427ad4223b3399d193b14bc09/users/449df1d62a9d4a6da2747c223dced904/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:08.858035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679ab2-06c8-4134-adaa-c8e4b0a1516b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:08.858318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679ab2-06c8-4134-adaa-c8e4b0a1516b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:08.858588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679ab2-06c8-4134-adaa-c8e4b0a1516b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:08.884229 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:08.887023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fd679ab2-06c8-4134-adaa-c8e4b0a1516b None None] MFA Rules not processed for user `449df1d62a9d4a6da2747c223dced904`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:08.888243 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fd679ab2-06c8-4134-adaa-c8e4b0a1516b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:08.952798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 942/1859] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:08.989076 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215b09b-f63b-4eb0-bb0a-cb3de72a7ed7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:09.031029 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:09.043538 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215b09b-f63b-4eb0-bb0a-cb3de72a7ed7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:09.047057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58071760-5e87-4bac-b9f6-c8151101ff59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:09.047657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58071760-5e87-4bac-b9f6-c8151101ff59 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:09.047935 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58071760-5e87-4bac-b9f6-c8151101ff59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:09.048204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58071760-5e87-4bac-b9f6-c8151101ff59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:09.056931 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58071760-5e87-4bac-b9f6-c8151101ff59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:09.058383 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58071760-5e87-4bac-b9f6-c8151101ff59 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:09.067556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 918/1860] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:08 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:09.084488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd679ab2-06c8-4134-adaa-c8e4b0a1516b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:09.148710 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd679ab2-06c8-4134-adaa-c8e4b0a1516b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:09.153238 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:09.153865 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:09.154142 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:09.154413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] PATH_INFO: `/v3/users/449df1d62a9d4a6da2747c223dced904` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:09.155534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] RBAC: Authorizing `identity:delete_user(user_id=449df1d62a9d4a6da2747c223dced904)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:09.157535 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:09.301342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '449df1d62a9d4a6da2747c223dced904'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:09.308826 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '449df1d62a9d4a6da2747c223dced904'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:09.309531 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '449df1d62a9d4a6da2747c223dced904'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:09.310148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '449df1d62a9d4a6da2747c223dced904'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:09.354872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '449df1d62a9d4a6da2747c223dced904'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:09.373005 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 943/1861] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:09 2022] DELETE /identity/v3/users/449df1d62a9d4a6da2747c223dced904 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:09.377540 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54d76-e3e1-4fb3-bae4-9617d5f29ca4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:09.377835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54d76-e3e1-4fb3-bae4-9617d5f29ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:09.378140 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54d76-e3e1-4fb3-bae4-9617d5f29ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:09.399013 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a0b54d76-e3e1-4fb3-bae4-9617d5f29ca4 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:09.400233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a0b54d76-e3e1-4fb3-bae4-9617d5f29ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:09.459682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 919/1862] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:09.469334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:09.470747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:09.497696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:09.528971 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca61435-6d7b-427a-a169-31d553a365be admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:09.532665 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b6de3-4708-43a8-8e16-b32d02caf630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:09.533308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b6de3-4708-43a8-8e16-b32d02caf630 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:09.533589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b6de3-4708-43a8-8e16-b32d02caf630 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:09.533859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b6de3-4708-43a8-8e16-b32d02caf630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:09.543559 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b6de3-4708-43a8-8e16-b32d02caf630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:09.544999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b6de3-4708-43a8-8e16-b32d02caf630 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:09.556156 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 944/1863] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:10.037322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54d76-e3e1-4fb3-bae4-9617d5f29ca4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:10.038995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a0b54d76-e3e1-4fb3-bae4-9617d5f29ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:10.052135 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54d76-e3e1-4fb3-bae4-9617d5f29ca4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:10.057568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:10.058222 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.058501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.058773 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] PATH_INFO: `/v3/projects/67baab2427ad4223b3399d193b14bc09` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.061314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67baab2427ad4223b3399d193b14bc09)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:10.063277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:10.183938 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67baab2427ad4223b3399d193b14bc09', 'request_id': 'req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:10.190348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67baab2427ad4223b3399d193b14bc09', 'request_id': 'req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:10.269806 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67baab2427ad4223b3399d193b14bc09', 'request_id': 'req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:10.280572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] The token cache is being invalidate because project 67baab2427ad4223b3399d193b14bc09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:10.280862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:10.335629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d6d9b339-07ce-44eb-82cf-92a9cf9c9810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:10.335629 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 920/1864] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:10 2022] DELETE /identity/v3/projects/67baab2427ad4223b3399d193b14bc09 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:10.336681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a848a-b62e-470f-babe-255f36332394 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.336681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a848a-b62e-470f-babe-255f36332394 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.336681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a848a-b62e-470f-babe-255f36332394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.336681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-940a848a-b62e-470f-babe-255f36332394 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:10.336681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-940a848a-b62e-470f-babe-255f36332394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:10.373438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 945/1865] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:10.382837 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646b482-22f9-473f-92d6-177ee15965ce None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.383087 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646b482-22f9-473f-92d6-177ee15965ce None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.383362 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646b482-22f9-473f-92d6-177ee15965ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.403965 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8646b482-22f9-473f-92d6-177ee15965ce None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:10.405178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8646b482-22f9-473f-92d6-177ee15965ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:10.413119 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 921/1866] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:10.417481 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a848a-b62e-470f-babe-255f36332394 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:10.427145 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a848a-b62e-470f-babe-255f36332394 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:10.431953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396fc13-695b-422d-b2b6-e9b87ee2148c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:10.432552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396fc13-695b-422d-b2b6-e9b87ee2148c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.432833 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396fc13-695b-422d-b2b6-e9b87ee2148c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.433112 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396fc13-695b-422d-b2b6-e9b87ee2148c admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.433822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396fc13-695b-422d-b2b6-e9b87ee2148c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:10.434127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396fc13-695b-422d-b2b6-e9b87ee2148c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:10.435731 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396fc13-695b-422d-b2b6-e9b87ee2148c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:10.448111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 946/1867] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:05:10.452294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646b482-22f9-473f-92d6-177ee15965ce None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:10.462610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646b482-22f9-473f-92d6-177ee15965ce None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:10.467325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ae6b0-ca01-4452-828e-b3d8b3c6b626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:10.467925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ae6b0-ca01-4452-828e-b3d8b3c6b626 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.468207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ae6b0-ca01-4452-828e-b3d8b3c6b626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.468484 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ae6b0-ca01-4452-828e-b3d8b3c6b626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.469128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ae6b0-ca01-4452-828e-b3d8b3c6b626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:10.471102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ae6b0-ca01-4452-828e-b3d8b3c6b626 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:10.568432 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 922/1868] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:10 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:10.572572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396fc13-695b-422d-b2b6-e9b87ee2148c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:10.583020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396fc13-695b-422d-b2b6-e9b87ee2148c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:10.643490 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb38d4-eaad-467b-941e-15d523a300e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:10.644013 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb38d4-eaad-467b-941e-15d523a300e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.644294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb38d4-eaad-467b-941e-15d523a300e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.644566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb38d4-eaad-467b-941e-15d523a300e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.645856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb38d4-eaad-467b-941e-15d523a300e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:10.647856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb38d4-eaad-467b-941e-15d523a300e6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:10.758689 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 947/1869] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:10 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:10.760200 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ae6b0-ca01-4452-828e-b3d8b3c6b626 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:10.770685 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ae6b0-ca01-4452-828e-b3d8b3c6b626 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:10.775240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9ba41-78d8-4b63-b777-239fc1f08219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:10.775833 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9ba41-78d8-4b63-b777-239fc1f08219 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.776111 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9ba41-78d8-4b63-b777-239fc1f08219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.776382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9ba41-78d8-4b63-b777-239fc1f08219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.777014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9ba41-78d8-4b63-b777-239fc1f08219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:10.778645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9ba41-78d8-4b63-b777-239fc1f08219 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:10.788303 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 923/1870] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:05:10.792204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb38d4-eaad-467b-941e-15d523a300e6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:10.803173 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb38d4-eaad-467b-941e-15d523a300e6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:10.807849 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262a0c8-599c-4d8f-9595-0706d2a586a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:10.808449 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262a0c8-599c-4d8f-9595-0706d2a586a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.808730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262a0c8-599c-4d8f-9595-0706d2a586a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.809080 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262a0c8-599c-4d8f-9595-0706d2a586a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.809760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262a0c8-599c-4d8f-9595-0706d2a586a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:10.811406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262a0c8-599c-4d8f-9595-0706d2a586a8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:10.822690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 948/1871] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:10.826699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9ba41-78d8-4b63-b777-239fc1f08219 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:10.836605 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9ba41-78d8-4b63-b777-239fc1f08219 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:10.841273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6d7f5-c079-41e1-8fef-4e05b705b653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:10.841927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6d7f5-c079-41e1-8fef-4e05b705b653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.842209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6d7f5-c079-41e1-8fef-4e05b705b653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.842489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6d7f5-c079-41e1-8fef-4e05b705b653 admin admin] PATH_INFO: `/v3/projects/39c7be98faab46b6a074998d358bb49a/users/4441b878701d48c686c08cce871b5e36/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.857054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6d7f5-c079-41e1-8fef-4e05b705b653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39c7be98faab46b6a074998d358bb49a, user_id=4441b878701d48c686c08cce871b5e36, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:10.859784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6d7f5-c079-41e1-8fef-4e05b705b653 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:10.933987 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 924/1872] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:10 2022] PUT /identity/v3/projects/39c7be98faab46b6a074998d358bb49a/users/4441b878701d48c686c08cce871b5e36/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:10.938937 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4572cee-c09e-4ca2-a976-47b3e47f10a0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:10.939220 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4572cee-c09e-4ca2-a976-47b3e47f10a0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:10.939526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4572cee-c09e-4ca2-a976-47b3e47f10a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:10.959232 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c4572cee-c09e-4ca2-a976-47b3e47f10a0 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:10.960427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c4572cee-c09e-4ca2-a976-47b3e47f10a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:10.987342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:11.023318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 949/1873] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:11.034095 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6d7f5-c079-41e1-8fef-4e05b705b653 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:11.036056 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3fb6d7f5-c079-41e1-8fef-4e05b705b653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:11.041592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:11.095250 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6d7f5-c079-41e1-8fef-4e05b705b653 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:11.099986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62c7bd-9b09-48fd-bfb9-142e4f4af0db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:11.100636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62c7bd-9b09-48fd-bfb9-142e4f4af0db service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:11.100973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62c7bd-9b09-48fd-bfb9-142e4f4af0db service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:11.101403 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62c7bd-9b09-48fd-bfb9-142e4f4af0db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:11.112078 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62c7bd-9b09-48fd-bfb9-142e4f4af0db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:11.114394 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62c7bd-9b09-48fd-bfb9-142e4f4af0db service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:11.124322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 925/1874] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:16.677136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7cf7d-03e1-4d2c-ae52-61f3d8bb1813 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:16.681437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7cf7d-03e1-4d2c-ae52-61f3d8bb1813 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:16.681437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7cf7d-03e1-4d2c-ae52-61f3d8bb1813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:16.709747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-aed7cf7d-03e1-4d2c-ae52-61f3d8bb1813 None None] MFA Rules not processed for user `4441b878701d48c686c08cce871b5e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:16.711079 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-aed7cf7d-03e1-4d2c-ae52-61f3d8bb1813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:16.789181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 950/1875] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:16.793779 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62c7bd-9b09-48fd-bfb9-142e4f4af0db service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:16.804435 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62c7bd-9b09-48fd-bfb9-142e4f4af0db service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:16.809233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba19d6-c935-46d0-8bae-5222cdf0eedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:16.809857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba19d6-c935-46d0-8bae-5222cdf0eedc admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:16.810143 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba19d6-c935-46d0-8bae-5222cdf0eedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:16.810420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba19d6-c935-46d0-8bae-5222cdf0eedc admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:16.811057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba19d6-c935-46d0-8bae-5222cdf0eedc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:16.812993 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba19d6-c935-46d0-8bae-5222cdf0eedc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:16.888566 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 926/1876] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:16 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:16.896905 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7cf7d-03e1-4d2c-ae52-61f3d8bb1813 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:16.908138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7cf7d-03e1-4d2c-ae52-61f3d8bb1813 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:16.977898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1ccb7-c2b2-4fc5-9f2a-188e1abe8c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:16.977898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1ccb7-c2b2-4fc5-9f2a-188e1abe8c9d admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:16.977898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1ccb7-c2b2-4fc5-9f2a-188e1abe8c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:16.977898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1ccb7-c2b2-4fc5-9f2a-188e1abe8c9d admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:16.978286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1ccb7-c2b2-4fc5-9f2a-188e1abe8c9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:16.978286 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1ccb7-c2b2-4fc5-9f2a-188e1abe8c9d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:17.071143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 951/1877] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:16 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:17.075263 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba19d6-c935-46d0-8bae-5222cdf0eedc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:17.093857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba19d6-c935-46d0-8bae-5222cdf0eedc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:17.099577 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61585705-ed0a-4c18-aad2-bde6672fed1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:17.100206 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61585705-ed0a-4c18-aad2-bde6672fed1c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:17.100489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61585705-ed0a-4c18-aad2-bde6672fed1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:17.100768 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61585705-ed0a-4c18-aad2-bde6672fed1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:17.101457 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61585705-ed0a-4c18-aad2-bde6672fed1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:17.103106 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61585705-ed0a-4c18-aad2-bde6672fed1c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:17.115754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 927/1878] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:17.120148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1ccb7-c2b2-4fc5-9f2a-188e1abe8c9d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:17.132909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1ccb7-c2b2-4fc5-9f2a-188e1abe8c9d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:17.137853 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd245f7b-a342-4755-84ad-723817a06849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:17.138560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245f7b-a342-4755-84ad-723817a06849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:17.141627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245f7b-a342-4755-84ad-723817a06849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:17.141627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245f7b-a342-4755-84ad-723817a06849 admin admin] PATH_INFO: `/v3/projects/0a18301279ce4591838c6faddd3c5389/users/dd0f84c3d4b448eca6944bc510ff3ab8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:17.156155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd245f7b-a342-4755-84ad-723817a06849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a18301279ce4591838c6faddd3c5389, user_id=dd0f84c3d4b448eca6944bc510ff3ab8, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:17.158917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd245f7b-a342-4755-84ad-723817a06849 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:17.283585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 952/1879] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:17 2022] PUT /identity/v3/projects/0a18301279ce4591838c6faddd3c5389/users/dd0f84c3d4b448eca6944bc510ff3ab8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:17.287335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61585705-ed0a-4c18-aad2-bde6672fed1c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:17.298175 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61585705-ed0a-4c18-aad2-bde6672fed1c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:17.362533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553313b-ba07-458e-812d-7ecd5aa74014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:17.363207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553313b-ba07-458e-812d-7ecd5aa74014 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:17.364044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553313b-ba07-458e-812d-7ecd5aa74014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:17.364044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553313b-ba07-458e-812d-7ecd5aa74014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:17.364637 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553313b-ba07-458e-812d-7ecd5aa74014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:17.366523 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553313b-ba07-458e-812d-7ecd5aa74014 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:17.381236 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 928/1880] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:05:17.382811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd245f7b-a342-4755-84ad-723817a06849 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:17.394007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd245f7b-a342-4755-84ad-723817a06849 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:17.400428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b61fe-167f-41b7-9dc3-a6609550d6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:17.401114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b61fe-167f-41b7-9dc3-a6609550d6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:17.401434 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b61fe-167f-41b7-9dc3-a6609550d6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:17.401708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b61fe-167f-41b7-9dc3-a6609550d6c0 admin admin] PATH_INFO: `/v3/projects/0a18301279ce4591838c6faddd3c5389/users/dd0f84c3d4b448eca6944bc510ff3ab8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:17.403659 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b61fe-167f-41b7-9dc3-a6609550d6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a18301279ce4591838c6faddd3c5389, user_id=dd0f84c3d4b448eca6944bc510ff3ab8, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:17.406512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b61fe-167f-41b7-9dc3-a6609550d6c0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:17.422325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 953/1881] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:17 2022] PUT /identity/v3/projects/0a18301279ce4591838c6faddd3c5389/users/dd0f84c3d4b448eca6944bc510ff3ab8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:23.038737 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e001d-a417-4e1d-add4-d4f42eb09d2c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:23.039229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e001d-a417-4e1d-add4-d4f42eb09d2c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:23.039340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e001d-a417-4e1d-add4-d4f42eb09d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:23.071739 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e82e001d-a417-4e1d-add4-d4f42eb09d2c None None] MFA Rules not processed for user `dd0f84c3d4b448eca6944bc510ff3ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:23.072906 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e82e001d-a417-4e1d-add4-d4f42eb09d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:23.135215 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:23.145872 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 929/1882] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:23.161717 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b61fe-167f-41b7-9dc3-a6609550d6c0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:23.164311 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bd2b61fe-167f-41b7-9dc3-a6609550d6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:23.185917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:23.258313 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b61fe-167f-41b7-9dc3-a6609550d6c0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:23.266479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da1901-cd8f-49da-99db-95e2e03d7dbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:23.267116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da1901-cd8f-49da-99db-95e2e03d7dbd service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:23.267397 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da1901-cd8f-49da-99db-95e2e03d7dbd service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:23.267668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da1901-cd8f-49da-99db-95e2e03d7dbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:23.335411 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da1901-cd8f-49da-99db-95e2e03d7dbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:23.337299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da1901-cd8f-49da-99db-95e2e03d7dbd service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:23.349413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 954/1883] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:25.870230 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4ac81-8a4d-46ec-80a5-5e82035b7f3b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:25.870771 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4ac81-8a4d-46ec-80a5-5e82035b7f3b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:25.870771 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4ac81-8a4d-46ec-80a5-5e82035b7f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:25.893010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d1d4ac81-8a4d-46ec-80a5-5e82035b7f3b None None] MFA Rules not processed for user `4441b878701d48c686c08cce871b5e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:25.894772 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d1d4ac81-8a4d-46ec-80a5-5e82035b7f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:25.906401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 930/1884] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:30.580313 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da1901-cd8f-49da-99db-95e2e03d7dbd service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:30.582966 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-34da1901-cd8f-49da-99db-95e2e03d7dbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:30.657607 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da1901-cd8f-49da-99db-95e2e03d7dbd service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:30.663039 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:30.663680 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:30.664040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:30.664362 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] PATH_INFO: `/v3/users/40e9b72b0d224e938e4b9d1539a50600` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:30.665483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e9b72b0d224e938e4b9d1539a50600)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:30.667646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:30.906996 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e9b72b0d224e938e4b9d1539a50600'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:30.912420 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:30.915356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e9b72b0d224e938e4b9d1539a50600'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:30.916084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e9b72b0d224e938e4b9d1539a50600'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:30.916687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e9b72b0d224e938e4b9d1539a50600'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:31.028435 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e9b72b0d224e938e4b9d1539a50600'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:31.051409 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 955/1885] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:30 2022] DELETE /identity/v3/users/40e9b72b0d224e938e4b9d1539a50600 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:31.056299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0cef3-557e-4534-beff-03fbe3387c44 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:31.062820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0cef3-557e-4534-beff-03fbe3387c44 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:31.062820 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0cef3-557e-4534-beff-03fbe3387c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:31.079585 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cab0cef3-557e-4534-beff-03fbe3387c44 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:31.080904 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cab0cef3-557e-4534-beff-03fbe3387c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:31.143126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 931/1886] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:31.158236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:31.159686 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:31.217880 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9d812-64bb-418f-b9a1-d89d2453d24b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:31.221458 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ad7b3-51ef-4c5e-b39a-1c2b9a5fe33f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:31.222065 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ad7b3-51ef-4c5e-b39a-1c2b9a5fe33f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:31.222344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ad7b3-51ef-4c5e-b39a-1c2b9a5fe33f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:31.222615 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ad7b3-51ef-4c5e-b39a-1c2b9a5fe33f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:31.232147 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ad7b3-51ef-4c5e-b39a-1c2b9a5fe33f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:31.233654 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ad7b3-51ef-4c5e-b39a-1c2b9a5fe33f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:31.243289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 956/1887] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:31.644840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0cef3-557e-4534-beff-03fbe3387c44 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:31.645082 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cab0cef3-557e-4534-beff-03fbe3387c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:31.659683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0cef3-557e-4534-beff-03fbe3387c44 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:31.666688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:31.667365 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:31.667655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:31.667941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] PATH_INFO: `/v3/projects/fe5c02aaba0c41a79346aed612e9a6a2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:31.669080 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5c02aaba0c41a79346aed612e9a6a2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:31.671052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:31.785598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5c02aaba0c41a79346aed612e9a6a2', 'request_id': 'req-4927904b-c1bc-4d34-b612-17adb960291b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:31.792711 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5c02aaba0c41a79346aed612e9a6a2', 'request_id': 'req-4927904b-c1bc-4d34-b612-17adb960291b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:31.868621 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5c02aaba0c41a79346aed612e9a6a2', 'request_id': 'req-4927904b-c1bc-4d34-b612-17adb960291b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:31.879740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] The token cache is being invalidate because project fe5c02aaba0c41a79346aed612e9a6a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:31.880031 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:31.880769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:31.882348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 932/1888] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:31 2022] DELETE /identity/v3/projects/fe5c02aaba0c41a79346aed612e9a6a2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:31.886332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ad7b3-51ef-4c5e-b39a-1c2b9a5fe33f service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:31.896038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ad7b3-51ef-4c5e-b39a-1c2b9a5fe33f service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:31.954787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:31.955348 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:31.955631 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:31.955908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] PATH_INFO: `/v3/users/21ce24adb54348db8dfaa1808a109533` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:31.957036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ce24adb54348db8dfaa1808a109533)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:31.959625 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:32.158832 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ce24adb54348db8dfaa1808a109533'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.167131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ce24adb54348db8dfaa1808a109533'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.168120 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ce24adb54348db8dfaa1808a109533'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.168869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ce24adb54348db8dfaa1808a109533'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.240339 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ce24adb54348db8dfaa1808a109533'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.254059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:32.265200 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 957/1889] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:31 2022] DELETE /identity/v3/users/21ce24adb54348db8dfaa1808a109533 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:32.688873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:32.690505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:32.705602 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927904b-c1bc-4d34-b612-17adb960291b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:32.771373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:32.772040 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:32.772323 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:32.772607 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] PATH_INFO: `/v3/projects/3c760bf4f65943df893ef12e6ab442c8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:32.773777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c760bf4f65943df893ef12e6ab442c8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:32.775796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:32.788321 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:32.913116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c760bf4f65943df893ef12e6ab442c8', 'request_id': 'req-39805528-53c7-4926-802c-47fe06224a92'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.919721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c760bf4f65943df893ef12e6ab442c8', 'request_id': 'req-39805528-53c7-4926-802c-47fe06224a92'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.983364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c760bf4f65943df893ef12e6ab442c8', 'request_id': 'req-39805528-53c7-4926-802c-47fe06224a92'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.995404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] The token cache is being invalidate because project 3c760bf4f65943df893ef12e6ab442c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:32.995699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.996332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:32.997915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 933/1890] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:32 2022] DELETE /identity/v3/projects/3c760bf4f65943df893ef12e6ab442c8 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:33.002009 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:33.003459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:33.017712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f7ba-98f5-4605-bca3-b13f1901c31b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:33.080737 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:33.081422 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:33.081706 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:33.081984 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] PATH_INFO: `/v3/users/6aef2896ad424858b5cca14044c9142c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:33.083251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aef2896ad424858b5cca14044c9142c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:33.085513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:33.341619 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aef2896ad424858b5cca14044c9142c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:33.350550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aef2896ad424858b5cca14044c9142c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:33.355731 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aef2896ad424858b5cca14044c9142c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:33.359393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aef2896ad424858b5cca14044c9142c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:33.434483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-020b3aa6-e052-40b7-9d22-51fc5dc7b781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aef2896ad424858b5cca14044c9142c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:33.453016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 958/1891] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:33 2022] DELETE /identity/v3/users/6aef2896ad424858b5cca14044c9142c => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:33.945378 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:33.946898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:33.961881 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39805528-53c7-4926-802c-47fe06224a92 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:34.024705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:34.025386 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:34.025668 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:34.025945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] PATH_INFO: `/v3/projects/fa934b86c1df4434b1e564404eccfd6f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:34.027097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa934b86c1df4434b1e564404eccfd6f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:34.029103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:34.206722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa934b86c1df4434b1e564404eccfd6f', 'request_id': 'req-fad534a4-1515-48e6-9bb6-282c51917faa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:34.213395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa934b86c1df4434b1e564404eccfd6f', 'request_id': 'req-fad534a4-1515-48e6-9bb6-282c51917faa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:34.314657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa934b86c1df4434b1e564404eccfd6f', 'request_id': 'req-fad534a4-1515-48e6-9bb6-282c51917faa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:34.326257 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] The token cache is being invalidate because project fa934b86c1df4434b1e564404eccfd6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:34.326547 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:34.327246 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fad534a4-1515-48e6-9bb6-282c51917faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:34.328721 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 934/1892] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:33 2022] DELETE /identity/v3/projects/fa934b86c1df4434b1e564404eccfd6f => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:34.334233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5ade9-5e4c-437e-946c-7c63caab22bf None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:34.334788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5ade9-5e4c-437e-946c-7c63caab22bf None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:34.334788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5ade9-5e4c-437e-946c-7c63caab22bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:34.355130 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-13a5ade9-5e4c-437e-946c-7c63caab22bf None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:34.356357 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-13a5ade9-5e4c-437e-946c-7c63caab22bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:34.420539 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 959/1893] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:34.430114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89de6c-87ab-48f3-8062-549a23e07a5d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:34.430398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89de6c-87ab-48f3-8062-549a23e07a5d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:34.430674 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89de6c-87ab-48f3-8062-549a23e07a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:34.451309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9c89de6c-87ab-48f3-8062-549a23e07a5d None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:34.452632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9c89de6c-87ab-48f3-8062-549a23e07a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:34.461161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 935/1894] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:34.465585 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5ade9-5e4c-437e-946c-7c63caab22bf None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:34.476036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5ade9-5e4c-437e-946c-7c63caab22bf None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:34.480856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cf60a-d8a8-4135-ab78-98947c19a733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:34.481514 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cf60a-d8a8-4135-ab78-98947c19a733 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:34.481787 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cf60a-d8a8-4135-ab78-98947c19a733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:34.482057 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cf60a-d8a8-4135-ab78-98947c19a733 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:34.482698 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cf60a-d8a8-4135-ab78-98947c19a733 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:34.483000 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cf60a-d8a8-4135-ab78-98947c19a733 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:34.484728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cf60a-d8a8-4135-ab78-98947c19a733 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:34.492515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:34.497292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 960/1895] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:05:34.501594 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89de6c-87ab-48f3-8062-549a23e07a5d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:34.510701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89de6c-87ab-48f3-8062-549a23e07a5d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:34.515525 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48db6de-dc35-4aeb-8258-83c97d0b56c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:34.516121 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48db6de-dc35-4aeb-8258-83c97d0b56c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:34.516398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48db6de-dc35-4aeb-8258-83c97d0b56c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:34.516670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48db6de-dc35-4aeb-8258-83c97d0b56c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:34.517391 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48db6de-dc35-4aeb-8258-83c97d0b56c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:34.519477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48db6de-dc35-4aeb-8258-83c97d0b56c6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:34.644363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 936/1896] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:34 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:34.648575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cf60a-d8a8-4135-ab78-98947c19a733 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:34.659733 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cf60a-d8a8-4135-ab78-98947c19a733 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:34.733736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf764237-dd49-43be-b643-3ee460dae203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:34.736695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf764237-dd49-43be-b643-3ee460dae203 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:34.736695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf764237-dd49-43be-b643-3ee460dae203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:34.736695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf764237-dd49-43be-b643-3ee460dae203 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:34.736695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf764237-dd49-43be-b643-3ee460dae203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:34.737502 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf764237-dd49-43be-b643-3ee460dae203 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:34.846254 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 961/1897] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:34 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:34.851169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48db6de-dc35-4aeb-8258-83c97d0b56c6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:34.856821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:34.862335 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48db6de-dc35-4aeb-8258-83c97d0b56c6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:34.867223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31a525-3f33-4775-a114-7f062341aa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:34.867716 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31a525-3f33-4775-a114-7f062341aa25 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:34.868019 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31a525-3f33-4775-a114-7f062341aa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:34.868294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31a525-3f33-4775-a114-7f062341aa25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:34.868969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31a525-3f33-4775-a114-7f062341aa25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:34.870660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31a525-3f33-4775-a114-7f062341aa25 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:34.880529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 937/1898] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:34.885281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf764237-dd49-43be-b643-3ee460dae203 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:34.895525 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf764237-dd49-43be-b643-3ee460dae203 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:34.901111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754a4e1-c3db-4478-be62-f63090539ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:34.901747 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754a4e1-c3db-4478-be62-f63090539ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:34.902031 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754a4e1-c3db-4478-be62-f63090539ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:34.902393 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754a4e1-c3db-4478-be62-f63090539ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:34.903088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754a4e1-c3db-4478-be62-f63090539ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:34.904695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754a4e1-c3db-4478-be62-f63090539ef6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:34.916566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 962/1899] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:34.921403 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31a525-3f33-4775-a114-7f062341aa25 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:34.933095 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31a525-3f33-4775-a114-7f062341aa25 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:34.937763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40681a9-5b33-49f3-b651-8b0130aed471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:34.938420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40681a9-5b33-49f3-b651-8b0130aed471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:34.938704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40681a9-5b33-49f3-b651-8b0130aed471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:34.938986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40681a9-5b33-49f3-b651-8b0130aed471 admin admin] PATH_INFO: `/v3/projects/d707e93c318640689ecb79078fbd18a1/users/cbe55ab08dc34cd8a97af5dfe8b1be45/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:34.954506 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40681a9-5b33-49f3-b651-8b0130aed471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d707e93c318640689ecb79078fbd18a1, user_id=cbe55ab08dc34cd8a97af5dfe8b1be45, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:34.957948 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40681a9-5b33-49f3-b651-8b0130aed471 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:35.053203 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 938/1900] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:34 2022] PUT /identity/v3/projects/d707e93c318640689ecb79078fbd18a1/users/cbe55ab08dc34cd8a97af5dfe8b1be45/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:35.064060 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bbecb-f3d4-4e49-ac36-c921ac977920 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:35.064348 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bbecb-f3d4-4e49-ac36-c921ac977920 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:35.064622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bbecb-f3d4-4e49-ac36-c921ac977920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:35.095258 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3d7bbecb-f3d4-4e49-ac36-c921ac977920 None None] MFA Rules not processed for user `cbe55ab08dc34cd8a97af5dfe8b1be45`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:35.096479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3d7bbecb-f3d4-4e49-ac36-c921ac977920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:35.168158 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 963/1901] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:35.180103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40681a9-5b33-49f3-b651-8b0130aed471 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:35.182069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f40681a9-5b33-49f3-b651-8b0130aed471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:35.245444 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40681a9-5b33-49f3-b651-8b0130aed471 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:35.251109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739a3a0-fce3-46b6-be01-d27f941fba22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:35.251109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739a3a0-fce3-46b6-be01-d27f941fba22 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:35.251109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739a3a0-fce3-46b6-be01-d27f941fba22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:35.251109 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739a3a0-fce3-46b6-be01-d27f941fba22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:35.263626 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739a3a0-fce3-46b6-be01-d27f941fba22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:35.265060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739a3a0-fce3-46b6-be01-d27f941fba22 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:35.275320 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 939/1902] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:35.732846 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bbecb-f3d4-4e49-ac36-c921ac977920 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:35.736761 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3d7bbecb-f3d4-4e49-ac36-c921ac977920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:35.821974 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bbecb-f3d4-4e49-ac36-c921ac977920 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:35.828707 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:35.829592 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:35.830312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:35.830878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] PATH_INFO: `/v3/users/c45853aaa3374114800a5f2957785126` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:35.832194 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45853aaa3374114800a5f2957785126)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:35.835577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:35.841710 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:36.061136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45853aaa3374114800a5f2957785126'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.069837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45853aaa3374114800a5f2957785126'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.070554 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45853aaa3374114800a5f2957785126'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.071271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45853aaa3374114800a5f2957785126'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.085829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739a3a0-fce3-46b6-be01-d27f941fba22 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:36.087331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4739a3a0-fce3-46b6-be01-d27f941fba22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:36.101982 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739a3a0-fce3-46b6-be01-d27f941fba22 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:36.115624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:36.116304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:36.116549 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:36.116836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] PATH_INFO: `/v3/users/b80b889cad234796943431cd6b3e80a8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:36.118233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b80b889cad234796943431cd6b3e80a8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:36.120132 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:36.155197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-65c5b5e0-0ec7-453b-a49a-405de11d3078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45853aaa3374114800a5f2957785126'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.175156 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 964/1903] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:35 2022] DELETE /identity/v3/users/c45853aaa3374114800a5f2957785126 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:36.179898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12951605-197b-441c-b839-8c9d31f808b1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:36.180320 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12951605-197b-441c-b839-8c9d31f808b1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:36.180603 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12951605-197b-441c-b839-8c9d31f808b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:36.200412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-12951605-197b-441c-b839-8c9d31f808b1 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:36.201650 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-12951605-197b-441c-b839-8c9d31f808b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:36.271959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 965/1904] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:36.281770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12951605-197b-441c-b839-8c9d31f808b1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:36.283178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-12951605-197b-441c-b839-8c9d31f808b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:36.344004 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b80b889cad234796943431cd6b3e80a8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.345514 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b80b889cad234796943431cd6b3e80a8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.354229 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b80b889cad234796943431cd6b3e80a8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.354499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12951605-197b-441c-b839-8c9d31f808b1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:36.355093 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b80b889cad234796943431cd6b3e80a8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.358228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bf791-0ee5-4052-83e4-255c82887eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:36.358898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bf791-0ee5-4052-83e4-255c82887eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:36.359189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bf791-0ee5-4052-83e4-255c82887eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:36.359854 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bf791-0ee5-4052-83e4-255c82887eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:36.361630 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-539bf791-0ee5-4052-83e4-255c82887eb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:36.374095 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bf791-0ee5-4052-83e4-255c82887eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:36.376812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bf791-0ee5-4052-83e4-255c82887eb1 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:36.385513 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 966/1905] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:05:36.417796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b80b889cad234796943431cd6b3e80a8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:36.447044 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 940/1906] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:36 2022] DELETE /identity/v3/users/b80b889cad234796943431cd6b3e80a8 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:36.451489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5b619-40f0-40d4-81fb-408bd73e6d4f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:36.451781 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5b619-40f0-40d4-81fb-408bd73e6d4f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:36.452058 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5b619-40f0-40d4-81fb-408bd73e6d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:36.477988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7eb5b619-40f0-40d4-81fb-408bd73e6d4f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:36.479266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7eb5b619-40f0-40d4-81fb-408bd73e6d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:36.514669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:36.550081 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 967/1907] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:36.567285 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:36.569468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:36.625905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:36.639589 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7fb72-f970-4248-b9dc-1bdfc6fd0a23 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:36.644344 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800d126-68ee-4db8-a106-7983ec409421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:36.644939 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800d126-68ee-4db8-a106-7983ec409421 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:36.645218 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800d126-68ee-4db8-a106-7983ec409421 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:36.645505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800d126-68ee-4db8-a106-7983ec409421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:36.655769 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800d126-68ee-4db8-a106-7983ec409421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:36.657405 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800d126-68ee-4db8-a106-7983ec409421 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:36.667156 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 941/1908] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:36.852795 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5b619-40f0-40d4-81fb-408bd73e6d4f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:36.854429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7eb5b619-40f0-40d4-81fb-408bd73e6d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:36.869350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5b619-40f0-40d4-81fb-408bd73e6d4f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:36.874057 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:36.874695 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:36.874998 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:36.875271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] PATH_INFO: `/v3/projects/b0923e472dbe4ae18809c2748a4ff9a1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:36.876389 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0923e472dbe4ae18809c2748a4ff9a1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:36.878429 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:37.037524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0923e472dbe4ae18809c2748a4ff9a1', 'request_id': 'req-97b483b9-7582-479e-8b83-c3a6e1b43f7e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.160670 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0923e472dbe4ae18809c2748a4ff9a1', 'request_id': 'req-97b483b9-7582-479e-8b83-c3a6e1b43f7e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.168515 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0923e472dbe4ae18809c2748a4ff9a1', 'request_id': 'req-97b483b9-7582-479e-8b83-c3a6e1b43f7e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.180538 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] The token cache is being invalidate because project b0923e472dbe4ae18809c2748a4ff9a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:37.180903 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.181564 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.183114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 968/1909] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:36 2022] DELETE /identity/v3/projects/b0923e472dbe4ae18809c2748a4ff9a1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:37.231524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800d126-68ee-4db8-a106-7983ec409421 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:37.231524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5800d126-68ee-4db8-a106-7983ec409421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:37.247448 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800d126-68ee-4db8-a106-7983ec409421 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:37.342840 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:37.347117 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:37.347704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:37.348204 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] PATH_INFO: `/v3/projects/36ac569f72814a7aac786d91843cb362` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:37.350506 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ac569f72814a7aac786d91843cb362)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:37.353023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:37.481023 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ac569f72814a7aac786d91843cb362', 'request_id': 'req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.487800 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ac569f72814a7aac786d91843cb362', 'request_id': 'req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.554815 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ac569f72814a7aac786d91843cb362', 'request_id': 'req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.569655 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] The token cache is being invalidate because project 36ac569f72814a7aac786d91843cb362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:37.570014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.570676 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.572161 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 942/1910] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:37 2022] DELETE /identity/v3/projects/36ac569f72814a7aac786d91843cb362 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:37.576089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:37.577712 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:37.590479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b483b9-7582-479e-8b83-c3a6e1b43f7e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:37.647771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:37.648413 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:37.648696 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:37.648970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] PATH_INFO: `/v3/users/3d73ebd47b4541cca800d71823d179c4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:37.650136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d73ebd47b4541cca800d71823d179c4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:37.652072 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:37.826324 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d73ebd47b4541cca800d71823d179c4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.831686 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:37.834160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d73ebd47b4541cca800d71823d179c4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.834821 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d73ebd47b4541cca800d71823d179c4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.835428 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d73ebd47b4541cca800d71823d179c4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.902736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ad83e9a3-d796-4fa9-b512-85eb31121563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d73ebd47b4541cca800d71823d179c4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:37.925921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 969/1911] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:37 2022] DELETE /identity/v3/users/3d73ebd47b4541cca800d71823d179c4 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:38.415134 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:38.421713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:38.438613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3ec00-3b70-4346-93d0-f6d3286de1f6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:38.504278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:38.504922 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:38.505228 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:38.506102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] PATH_INFO: `/v3/projects/9439a2c8c69e4c3db29d78f1e61450fa` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:38.507221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9439a2c8c69e4c3db29d78f1e61450fa)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:38.509133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:38.525719 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:38.652329 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9439a2c8c69e4c3db29d78f1e61450fa', 'request_id': 'req-51eca3b6-f9ca-4154-955e-b98b1243fd18'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:38.660039 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9439a2c8c69e4c3db29d78f1e61450fa', 'request_id': 'req-51eca3b6-f9ca-4154-955e-b98b1243fd18'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:38.795245 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9439a2c8c69e4c3db29d78f1e61450fa', 'request_id': 'req-51eca3b6-f9ca-4154-955e-b98b1243fd18'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:38.810976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] The token cache is being invalidate because project 9439a2c8c69e4c3db29d78f1e61450fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:38.811265 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:38.811912 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-51eca3b6-f9ca-4154-955e-b98b1243fd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:38.813500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 943/1912] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:38 2022] DELETE /identity/v3/projects/9439a2c8c69e4c3db29d78f1e61450fa => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:38.818624 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ebd01-52a2-4ead-8002-c926ead991ab None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:38.818908 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ebd01-52a2-4ead-8002-c926ead991ab None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:38.819184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ebd01-52a2-4ead-8002-c926ead991ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:38.839866 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-329ebd01-52a2-4ead-8002-c926ead991ab None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:38.841366 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-329ebd01-52a2-4ead-8002-c926ead991ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:38.910489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 970/1913] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:38.918906 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72524111-4a55-4e97-9018-c2e53333fa16 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:38.919188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72524111-4a55-4e97-9018-c2e53333fa16 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:38.919466 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72524111-4a55-4e97-9018-c2e53333fa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:38.941458 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-72524111-4a55-4e97-9018-c2e53333fa16 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:38.942575 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-72524111-4a55-4e97-9018-c2e53333fa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:38.951025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 944/1914] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:38.955369 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ebd01-52a2-4ead-8002-c926ead991ab None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:38.966617 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ebd01-52a2-4ead-8002-c926ead991ab None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:38.972059 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81737697-e867-4a0d-8dd1-5bd99a703486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:38.972694 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81737697-e867-4a0d-8dd1-5bd99a703486 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:38.972976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81737697-e867-4a0d-8dd1-5bd99a703486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:38.973332 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81737697-e867-4a0d-8dd1-5bd99a703486 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:38.974673 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81737697-e867-4a0d-8dd1-5bd99a703486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:38.975117 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81737697-e867-4a0d-8dd1-5bd99a703486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:38.976779 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81737697-e867-4a0d-8dd1-5bd99a703486 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:38.989316 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 971/1915] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:05:38.994305 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72524111-4a55-4e97-9018-c2e53333fa16 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:39.003420 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72524111-4a55-4e97-9018-c2e53333fa16 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:39.008225 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc971f6-a9c6-48af-aaf5-a47f4cca0dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:39.008821 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc971f6-a9c6-48af-aaf5-a47f4cca0dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:39.009100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc971f6-a9c6-48af-aaf5-a47f4cca0dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:39.009407 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc971f6-a9c6-48af-aaf5-a47f4cca0dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:39.010167 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc971f6-a9c6-48af-aaf5-a47f4cca0dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:39.012060 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc971f6-a9c6-48af-aaf5-a47f4cca0dd8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:39.122673 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 945/1916] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:38 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:39.127021 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81737697-e867-4a0d-8dd1-5bd99a703486 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:39.137427 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81737697-e867-4a0d-8dd1-5bd99a703486 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:39.194082 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7245d0-e05a-4dfe-8330-f0d0946ccccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:39.194618 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7245d0-e05a-4dfe-8330-f0d0946ccccd admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:39.194906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7245d0-e05a-4dfe-8330-f0d0946ccccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:39.195186 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7245d0-e05a-4dfe-8330-f0d0946ccccd admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:39.195826 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7245d0-e05a-4dfe-8330-f0d0946ccccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:39.197807 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7245d0-e05a-4dfe-8330-f0d0946ccccd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:39.297007 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 972/1917] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:39 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:39.301207 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc971f6-a9c6-48af-aaf5-a47f4cca0dd8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:39.312416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc971f6-a9c6-48af-aaf5-a47f4cca0dd8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:39.317036 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e2b39-6fa0-476b-bf3a-81b2772159b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:39.317680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e2b39-6fa0-476b-bf3a-81b2772159b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:39.318568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e2b39-6fa0-476b-bf3a-81b2772159b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:39.318915 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e2b39-6fa0-476b-bf3a-81b2772159b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:39.319600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e2b39-6fa0-476b-bf3a-81b2772159b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:39.321210 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e2b39-6fa0-476b-bf3a-81b2772159b8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:39.331411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 946/1918] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:39.336579 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7245d0-e05a-4dfe-8330-f0d0946ccccd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:39.346884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7245d0-e05a-4dfe-8330-f0d0946ccccd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:39.351839 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46803b46-558c-4da5-b9b5-829988316181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:39.352440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46803b46-558c-4da5-b9b5-829988316181 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:39.352719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46803b46-558c-4da5-b9b5-829988316181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:39.352991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46803b46-558c-4da5-b9b5-829988316181 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:39.353660 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46803b46-558c-4da5-b9b5-829988316181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:39.355259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46803b46-558c-4da5-b9b5-829988316181 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:39.366327 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 973/1919] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:39.370441 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e2b39-6fa0-476b-bf3a-81b2772159b8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:39.381221 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e2b39-6fa0-476b-bf3a-81b2772159b8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:39.385777 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99465a94-7247-48dc-9ef9-be8c2bf2fe86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:39.386418 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99465a94-7247-48dc-9ef9-be8c2bf2fe86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:39.386704 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99465a94-7247-48dc-9ef9-be8c2bf2fe86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:39.386980 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99465a94-7247-48dc-9ef9-be8c2bf2fe86 admin admin] PATH_INFO: `/v3/projects/8b0adb2e2da54c9abb58d8addd57f237/users/cf21f575d13b4fb6b45215a80baab996/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:39.407238 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99465a94-7247-48dc-9ef9-be8c2bf2fe86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b0adb2e2da54c9abb58d8addd57f237, user_id=cf21f575d13b4fb6b45215a80baab996, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:39.410002 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99465a94-7247-48dc-9ef9-be8c2bf2fe86 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:39.509614 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 947/1920] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:39 2022] PUT /identity/v3/projects/8b0adb2e2da54c9abb58d8addd57f237/users/cf21f575d13b4fb6b45215a80baab996/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:39.533138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75217-5141-4669-9dcd-2ef5b11dddca None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:39.533138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75217-5141-4669-9dcd-2ef5b11dddca None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:39.533138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75217-5141-4669-9dcd-2ef5b11dddca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:39.551694 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-cfc75217-5141-4669-9dcd-2ef5b11dddca None None] MFA Rules not processed for user `cf21f575d13b4fb6b45215a80baab996`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:39.554070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cfc75217-5141-4669-9dcd-2ef5b11dddca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:39.584529 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:39.624421 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 974/1921] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:39.630343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99465a94-7247-48dc-9ef9-be8c2bf2fe86 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:39.692665 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99465a94-7247-48dc-9ef9-be8c2bf2fe86 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:39.697756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1568e-5ee3-46bc-8057-8ec32dba516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:39.698395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1568e-5ee3-46bc-8057-8ec32dba516d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:39.698602 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1568e-5ee3-46bc-8057-8ec32dba516d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:39.698893 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1568e-5ee3-46bc-8057-8ec32dba516d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:39.699565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1568e-5ee3-46bc-8057-8ec32dba516d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:39.701569 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1568e-5ee3-46bc-8057-8ec32dba516d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:39.814539 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 948/1922] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:39 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:39.820193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc75217-5141-4669-9dcd-2ef5b11dddca None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:39.831199 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc75217-5141-4669-9dcd-2ef5b11dddca None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:39.886878 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c368b-8605-4696-939e-2914ff7d823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:39.887489 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c368b-8605-4696-939e-2914ff7d823c admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:39.887769 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c368b-8605-4696-939e-2914ff7d823c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:39.888041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c368b-8605-4696-939e-2914ff7d823c admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:39.888691 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c368b-8605-4696-939e-2914ff7d823c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:39.890697 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c368b-8605-4696-939e-2914ff7d823c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:39.971492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 975/1923] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:39 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:39.976635 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1568e-5ee3-46bc-8057-8ec32dba516d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:39.987406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1568e-5ee3-46bc-8057-8ec32dba516d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:39.992192 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9473a3d-bc3c-4c95-93f7-0d9e5ab51671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:39.992854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9473a3d-bc3c-4c95-93f7-0d9e5ab51671 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:39.993132 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9473a3d-bc3c-4c95-93f7-0d9e5ab51671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:39.993427 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9473a3d-bc3c-4c95-93f7-0d9e5ab51671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:39.994076 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9473a3d-bc3c-4c95-93f7-0d9e5ab51671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:39.995845 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9473a3d-bc3c-4c95-93f7-0d9e5ab51671 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:40.010565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 949/1924] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:40.014744 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c368b-8605-4696-939e-2914ff7d823c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:40.025779 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c368b-8605-4696-939e-2914ff7d823c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:40.030520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de4b24-3a8f-464b-b814-c63d14e7ba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:40.031239 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de4b24-3a8f-464b-b814-c63d14e7ba8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:40.031526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de4b24-3a8f-464b-b814-c63d14e7ba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:40.031804 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de4b24-3a8f-464b-b814-c63d14e7ba8f admin admin] PATH_INFO: `/v3/projects/e645bc7a719e40a6a7672cadfd6c2623/users/90473e57ed324066a6528387233191f8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:40.047589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de4b24-3a8f-464b-b814-c63d14e7ba8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e645bc7a719e40a6a7672cadfd6c2623, user_id=90473e57ed324066a6528387233191f8, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:40.050365 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de4b24-3a8f-464b-b814-c63d14e7ba8f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:40.160303 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 976/1925] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:40 2022] PUT /identity/v3/projects/e645bc7a719e40a6a7672cadfd6c2623/users/90473e57ed324066a6528387233191f8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:40.165169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9473a3d-bc3c-4c95-93f7-0d9e5ab51671 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:40.176147 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9473a3d-bc3c-4c95-93f7-0d9e5ab51671 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:40.189066 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:40.236083 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5f635-2973-45fd-8382-48bb1c7eb75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:40.236705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5f635-2973-45fd-8382-48bb1c7eb75b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:40.236989 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5f635-2973-45fd-8382-48bb1c7eb75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:40.237299 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5f635-2973-45fd-8382-48bb1c7eb75b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:40.237952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5f635-2973-45fd-8382-48bb1c7eb75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:40.239558 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5f635-2973-45fd-8382-48bb1c7eb75b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:40.252722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 950/1926] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:40.253889 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de4b24-3a8f-464b-b814-c63d14e7ba8f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:40.265930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de4b24-3a8f-464b-b814-c63d14e7ba8f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:40.271037 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a931d7-0526-4d4c-b3ad-23b2625dfe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:40.272597 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a931d7-0526-4d4c-b3ad-23b2625dfe00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:40.272888 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a931d7-0526-4d4c-b3ad-23b2625dfe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:40.273166 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a931d7-0526-4d4c-b3ad-23b2625dfe00 admin admin] PATH_INFO: `/v3/projects/e645bc7a719e40a6a7672cadfd6c2623/users/90473e57ed324066a6528387233191f8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:40.275134 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a931d7-0526-4d4c-b3ad-23b2625dfe00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e645bc7a719e40a6a7672cadfd6c2623, user_id=90473e57ed324066a6528387233191f8, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:40.277846 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a931d7-0526-4d4c-b3ad-23b2625dfe00 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:40.288743 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 977/1927] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:40 2022] PUT /identity/v3/projects/e645bc7a719e40a6a7672cadfd6c2623/users/90473e57ed324066a6528387233191f8/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:40.299041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7dca1-ca0d-4852-9cae-cf6283d81c98 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:40.299356 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7dca1-ca0d-4852-9cae-cf6283d81c98 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:40.299636 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7dca1-ca0d-4852-9cae-cf6283d81c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:40.334706 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-07b7dca1-ca0d-4852-9cae-cf6283d81c98 None None] MFA Rules not processed for user `90473e57ed324066a6528387233191f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:40.335918 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-07b7dca1-ca0d-4852-9cae-cf6283d81c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:40.408735 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 951/1928] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:40.419450 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a931d7-0526-4d4c-b3ad-23b2625dfe00 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:40.421168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-98a931d7-0526-4d4c-b3ad-23b2625dfe00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:40.482325 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a931d7-0526-4d4c-b3ad-23b2625dfe00 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:40.486190 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a724064-9a2b-4a57-ad31-948ffade0b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:40.486797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724064-9a2b-4a57-ad31-948ffade0b2d service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:40.487075 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724064-9a2b-4a57-ad31-948ffade0b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:40.487344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724064-9a2b-4a57-ad31-948ffade0b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:40.496269 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a724064-9a2b-4a57-ad31-948ffade0b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:40.497732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a724064-9a2b-4a57-ad31-948ffade0b2d service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:40.507911 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 978/1929] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:43.421027 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7dca1-ca0d-4852-9cae-cf6283d81c98 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:43.483652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7dca1-ca0d-4852-9cae-cf6283d81c98 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:43.488162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:43.488797 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:43.489078 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:43.489387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] PATH_INFO: `/v3/users/cf21f575d13b4fb6b45215a80baab996` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:43.490740 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf21f575d13b4fb6b45215a80baab996)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:43.492664 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:43.683459 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:43.689014 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf21f575d13b4fb6b45215a80baab996'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:43.689699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf21f575d13b4fb6b45215a80baab996'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:43.697454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf21f575d13b4fb6b45215a80baab996'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:43.698102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf21f575d13b4fb6b45215a80baab996'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:43.799884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf21f575d13b4fb6b45215a80baab996'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:43.825829 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 952/1930] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:43 2022] DELETE /identity/v3/users/cf21f575d13b4fb6b45215a80baab996 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:43.830162 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a024b0-881e-4e31-9336-47b5a9c4680f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:43.830628 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a024b0-881e-4e31-9336-47b5a9c4680f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:43.830722 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a024b0-881e-4e31-9336-47b5a9c4680f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:43.852407 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-22a024b0-881e-4e31-9336-47b5a9c4680f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:43.853660 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-22a024b0-881e-4e31-9336-47b5a9c4680f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:43.871261 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:43.913131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 979/1931] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:43.923066 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:43.924617 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:43.984962 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227884e9-88d9-4689-aaf4-105bb7479288 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:43.988602 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f061036-7ace-4831-a7ac-06f8a17f96da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:43.989248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f061036-7ace-4831-a7ac-06f8a17f96da service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:43.990091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f061036-7ace-4831-a7ac-06f8a17f96da service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:43.991761 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f061036-7ace-4831-a7ac-06f8a17f96da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:43.999609 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f061036-7ace-4831-a7ac-06f8a17f96da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:44.001045 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f061036-7ace-4831-a7ac-06f8a17f96da service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:44.010842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 953/1932] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:44.465241 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a024b0-881e-4e31-9336-47b5a9c4680f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:44.466705 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-22a024b0-881e-4e31-9336-47b5a9c4680f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:44.479896 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a024b0-881e-4e31-9336-47b5a9c4680f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:44.485148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:44.485822 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:44.486104 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:44.486381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] PATH_INFO: `/v3/projects/8b0adb2e2da54c9abb58d8addd57f237` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:44.487552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b0adb2e2da54c9abb58d8addd57f237)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:44.489560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:44.602528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b0adb2e2da54c9abb58d8addd57f237', 'request_id': 'req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:44.674373 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b0adb2e2da54c9abb58d8addd57f237', 'request_id': 'req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:44.682094 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b0adb2e2da54c9abb58d8addd57f237', 'request_id': 'req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:44.694191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] The token cache is being invalidate because project 8b0adb2e2da54c9abb58d8addd57f237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:44.694663 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:44.695550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:44.697970 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 980/1933] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:44 2022] DELETE /identity/v3/projects/8b0adb2e2da54c9abb58d8addd57f237 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:44.701945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f061036-7ace-4831-a7ac-06f8a17f96da service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:44.712394 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f061036-7ace-4831-a7ac-06f8a17f96da service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:44.772774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:44.773449 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:44.773726 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:44.773996 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] PATH_INFO: `/v3/users/90473e57ed324066a6528387233191f8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:44.776162 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90473e57ed324066a6528387233191f8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:44.778141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:44.946544 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90473e57ed324066a6528387233191f8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:44.947219 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90473e57ed324066a6528387233191f8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:44.954739 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90473e57ed324066a6528387233191f8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:44.955389 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90473e57ed324066a6528387233191f8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:44.959093 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:45.017168 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48c047f0-7152-4c3c-b466-f6df60c5a8c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90473e57ed324066a6528387233191f8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:45.037054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 954/1934] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:44 2022] DELETE /identity/v3/users/90473e57ed324066a6528387233191f8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:45.485816 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:45.490296 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:45.500745 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27aeb1f-b743-4e2f-8dfb-bb414b11d9c7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:45.569912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:45.570560 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:45.570839 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:45.571116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] PATH_INFO: `/v3/projects/e645bc7a719e40a6a7672cadfd6c2623` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:45.572232 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e645bc7a719e40a6a7672cadfd6c2623)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:45.574209 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:45.652831 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:45.654308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e645bc7a719e40a6a7672cadfd6c2623', 'request_id': 'req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:45.714205 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e645bc7a719e40a6a7672cadfd6c2623', 'request_id': 'req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:45.721155 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e645bc7a719e40a6a7672cadfd6c2623', 'request_id': 'req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:45.732415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] The token cache is being invalidate because project e645bc7a719e40a6a7672cadfd6c2623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:45.732705 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:45.733388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0b2f2207-0e95-47ed-a5b0-6c1cabfc2f27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:45.734876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 981/1935] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:45 2022] DELETE /identity/v3/projects/e645bc7a719e40a6a7672cadfd6c2623 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:45.739911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c32bef-9047-43ae-9193-640b2a8507e9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:45.740382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c32bef-9047-43ae-9193-640b2a8507e9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:45.740470 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c32bef-9047-43ae-9193-640b2a8507e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:45.760973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a1c32bef-9047-43ae-9193-640b2a8507e9 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:45.762233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a1c32bef-9047-43ae-9193-640b2a8507e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:45.823842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 955/1936] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:45.831782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07a500-e5e7-408c-991d-75678d81c23f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:45.832330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07a500-e5e7-408c-991d-75678d81c23f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:45.832669 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07a500-e5e7-408c-991d-75678d81c23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:45.854803 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5f07a500-e5e7-408c-991d-75678d81c23f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:45.857384 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5f07a500-e5e7-408c-991d-75678d81c23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:45.866744 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 982/1937] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:45.872247 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c32bef-9047-43ae-9193-640b2a8507e9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:45.882703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c32bef-9047-43ae-9193-640b2a8507e9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:45.887371 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed625f-860a-4844-9a1d-77e5d930be9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:45.887973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed625f-860a-4844-9a1d-77e5d930be9b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:45.888253 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed625f-860a-4844-9a1d-77e5d930be9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:45.888532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed625f-860a-4844-9a1d-77e5d930be9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:45.889164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed625f-860a-4844-9a1d-77e5d930be9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:45.889510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed625f-860a-4844-9a1d-77e5d930be9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:45.891097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed625f-860a-4844-9a1d-77e5d930be9b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:45.902725 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 956/1938] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:05:45.907096 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07a500-e5e7-408c-991d-75678d81c23f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:45.916138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07a500-e5e7-408c-991d-75678d81c23f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:45.920782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2b407-6a7f-412b-baed-f68fe0cfcb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:45.921424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2b407-6a7f-412b-baed-f68fe0cfcb7c admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:45.921703 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2b407-6a7f-412b-baed-f68fe0cfcb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:45.921975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2b407-6a7f-412b-baed-f68fe0cfcb7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:45.922602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2b407-6a7f-412b-baed-f68fe0cfcb7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:45.924496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2b407-6a7f-412b-baed-f68fe0cfcb7c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:46.018568 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 983/1939] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:45 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:46.022935 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed625f-860a-4844-9a1d-77e5d930be9b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:46.033256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed625f-860a-4844-9a1d-77e5d930be9b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:46.092061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa5060-78da-4c76-8a35-4f7e7bdffc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:46.092680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa5060-78da-4c76-8a35-4f7e7bdffc6b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:46.092964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa5060-78da-4c76-8a35-4f7e7bdffc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:46.093283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa5060-78da-4c76-8a35-4f7e7bdffc6b admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:46.093930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa5060-78da-4c76-8a35-4f7e7bdffc6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:46.095857 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa5060-78da-4c76-8a35-4f7e7bdffc6b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:46.216776 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 957/1940] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:46 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:05:46.220869 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2b407-6a7f-412b-baed-f68fe0cfcb7c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:46.232507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2b407-6a7f-412b-baed-f68fe0cfcb7c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:46.237153 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925a62d-fef7-489c-8de4-d448160a5e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:46.237797 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925a62d-fef7-489c-8de4-d448160a5e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:46.238078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925a62d-fef7-489c-8de4-d448160a5e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:46.238353 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925a62d-fef7-489c-8de4-d448160a5e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:46.238994 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925a62d-fef7-489c-8de4-d448160a5e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:46.241360 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925a62d-fef7-489c-8de4-d448160a5e82 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:46.251437 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 984/1941] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:46.255501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa5060-78da-4c76-8a35-4f7e7bdffc6b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:46.265517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa5060-78da-4c76-8a35-4f7e7bdffc6b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:46.270228 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e0770-9086-4c79-a873-7620a3bcc810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:46.270973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e0770-9086-4c79-a873-7620a3bcc810 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:46.271452 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e0770-9086-4c79-a873-7620a3bcc810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:46.271683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e0770-9086-4c79-a873-7620a3bcc810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:46.272279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e0770-9086-4c79-a873-7620a3bcc810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:46.280780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e0770-9086-4c79-a873-7620a3bcc810 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:46.286680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 958/1942] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:46.290867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925a62d-fef7-489c-8de4-d448160a5e82 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:46.301227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925a62d-fef7-489c-8de4-d448160a5e82 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:46.307026 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3e406-578d-4e78-9ff5-4fab494ecb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:46.307728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3e406-578d-4e78-9ff5-4fab494ecb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:46.308035 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3e406-578d-4e78-9ff5-4fab494ecb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:46.308311 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3e406-578d-4e78-9ff5-4fab494ecb85 admin admin] PATH_INFO: `/v3/projects/3701ad3848cd4600b76c41c59fa17401/users/40b0cd94bb1d4837baede8ab5532c593/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:46.324711 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3e406-578d-4e78-9ff5-4fab494ecb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3701ad3848cd4600b76c41c59fa17401, user_id=40b0cd94bb1d4837baede8ab5532c593, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:46.327555 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3e406-578d-4e78-9ff5-4fab494ecb85 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:46.413054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 985/1943] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:46 2022] PUT /identity/v3/projects/3701ad3848cd4600b76c41c59fa17401/users/40b0cd94bb1d4837baede8ab5532c593/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:46.423124 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3e621-a4dd-4ebe-9989-a1410bb65bd6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:46.423412 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3e621-a4dd-4ebe-9989-a1410bb65bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:46.423688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3e621-a4dd-4ebe-9989-a1410bb65bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:46.452986 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2ab3e621-a4dd-4ebe-9989-a1410bb65bd6 None None] MFA Rules not processed for user `40b0cd94bb1d4837baede8ab5532c593`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:46.454240 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2ab3e621-a4dd-4ebe-9989-a1410bb65bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:46.493180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:46.539556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 959/1944] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:46.541614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3e406-578d-4e78-9ff5-4fab494ecb85 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:46.613844 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3e406-578d-4e78-9ff5-4fab494ecb85 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:46.619403 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e443300-226e-4bb7-bf62-3befc1e8d56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:46.620030 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e443300-226e-4bb7-bf62-3befc1e8d56e admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:46.620452 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e443300-226e-4bb7-bf62-3befc1e8d56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:46.620740 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e443300-226e-4bb7-bf62-3befc1e8d56e admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:46.623666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e443300-226e-4bb7-bf62-3befc1e8d56e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:46.623666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e443300-226e-4bb7-bf62-3befc1e8d56e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:46.691070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 986/1945] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:46 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:46.692191 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3e621-a4dd-4ebe-9989-a1410bb65bd6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:46.703284 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3e621-a4dd-4ebe-9989-a1410bb65bd6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:46.760816 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa495-bf29-417f-87be-0378b346ae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:46.761500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa495-bf29-417f-87be-0378b346ae3a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:46.761694 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa495-bf29-417f-87be-0378b346ae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:46.761973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa495-bf29-417f-87be-0378b346ae3a admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:46.762708 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fa495-bf29-417f-87be-0378b346ae3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:46.764652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fa495-bf29-417f-87be-0378b346ae3a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:46.834533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 960/1946] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:46 2022] POST /identity/v3/users => generated 297 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:05:46.838811 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e443300-226e-4bb7-bf62-3befc1e8d56e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:46.850338 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e443300-226e-4bb7-bf62-3befc1e8d56e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:46.855236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96527658-50b8-4b32-b97f-edaaae160f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:46.855953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96527658-50b8-4b32-b97f-edaaae160f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:46.856236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96527658-50b8-4b32-b97f-edaaae160f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:46.856558 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96527658-50b8-4b32-b97f-edaaae160f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:46.857235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96527658-50b8-4b32-b97f-edaaae160f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:46.858861 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96527658-50b8-4b32-b97f-edaaae160f2e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:46.870512 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 987/1947] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:46.874830 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa495-bf29-417f-87be-0378b346ae3a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:46.886123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa495-bf29-417f-87be-0378b346ae3a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:46.890950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99a3bd-8afb-40dd-ac4a-4c0e17b845e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:46.891604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99a3bd-8afb-40dd-ac4a-4c0e17b845e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:46.891882 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99a3bd-8afb-40dd-ac4a-4c0e17b845e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:46.892154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99a3bd-8afb-40dd-ac4a-4c0e17b845e7 admin admin] PATH_INFO: `/v3/projects/a0d1522c804448ffb9b74c9d26013b53/users/a2a0870ce7b343caaf36fdcc597712aa/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:46.907610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99a3bd-8afb-40dd-ac4a-4c0e17b845e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d1522c804448ffb9b74c9d26013b53, user_id=a2a0870ce7b343caaf36fdcc597712aa, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:46.911293 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99a3bd-8afb-40dd-ac4a-4c0e17b845e7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:46.995854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 961/1948] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:46 2022] PUT /identity/v3/projects/a0d1522c804448ffb9b74c9d26013b53/users/a2a0870ce7b343caaf36fdcc597712aa/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:47.000008 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96527658-50b8-4b32-b97f-edaaae160f2e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:47.010075 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96527658-50b8-4b32-b97f-edaaae160f2e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:47.054197 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:47.067898 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcaff0-a324-4f9e-b4d4-fc28d0588e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:47.068519 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcaff0-a324-4f9e-b4d4-fc28d0588e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:47.068803 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcaff0-a324-4f9e-b4d4-fc28d0588e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:47.069085 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcaff0-a324-4f9e-b4d4-fc28d0588e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:47.069788 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dcaff0-a324-4f9e-b4d4-fc28d0588e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:47.071383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dcaff0-a324-4f9e-b4d4-fc28d0588e97 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:47.081496 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 988/1949] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:47.085916 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99a3bd-8afb-40dd-ac4a-4c0e17b845e7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:47.096235 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99a3bd-8afb-40dd-ac4a-4c0e17b845e7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:47.101454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451a013-5c12-4dc3-b3a7-d54e08973115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:47.102110 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451a013-5c12-4dc3-b3a7-d54e08973115 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:47.102392 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451a013-5c12-4dc3-b3a7-d54e08973115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:47.102669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451a013-5c12-4dc3-b3a7-d54e08973115 admin admin] PATH_INFO: `/v3/projects/a0d1522c804448ffb9b74c9d26013b53/users/a2a0870ce7b343caaf36fdcc597712aa/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:47.104572 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451a013-5c12-4dc3-b3a7-d54e08973115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d1522c804448ffb9b74c9d26013b53, user_id=a2a0870ce7b343caaf36fdcc597712aa, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:47.107267 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451a013-5c12-4dc3-b3a7-d54e08973115 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:47.118578 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 962/1950] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:47 2022] PUT /identity/v3/projects/a0d1522c804448ffb9b74c9d26013b53/users/a2a0870ce7b343caaf36fdcc597712aa/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:47.128888 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9a076-9b8d-4d27-a5c8-ca448f19cdba None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:47.129182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9a076-9b8d-4d27-a5c8-ca448f19cdba None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:47.129499 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9a076-9b8d-4d27-a5c8-ca448f19cdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:47.157025 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9df9a076-9b8d-4d27-a5c8-ca448f19cdba None None] MFA Rules not processed for user `a2a0870ce7b343caaf36fdcc597712aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:47.158319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9df9a076-9b8d-4d27-a5c8-ca448f19cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:47.227668 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 989/1951] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:47.238428 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451a013-5c12-4dc3-b3a7-d54e08973115 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:47.240298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8451a013-5c12-4dc3-b3a7-d54e08973115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:47.299413 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451a013-5c12-4dc3-b3a7-d54e08973115 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:47.303356 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a53cfe-043d-4aa3-9e17-c1a36068b742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:47.303964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a53cfe-043d-4aa3-9e17-c1a36068b742 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:47.304294 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a53cfe-043d-4aa3-9e17-c1a36068b742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:47.304519 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a53cfe-043d-4aa3-9e17-c1a36068b742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:47.313467 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a53cfe-043d-4aa3-9e17-c1a36068b742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:47.314898 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a53cfe-043d-4aa3-9e17-c1a36068b742 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:47.324950 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 963/1952] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:47.430344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9a076-9b8d-4d27-a5c8-ca448f19cdba None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:47.440419 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9a076-9b8d-4d27-a5c8-ca448f19cdba None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:47.444741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc4603-b3b9-4e42-b343-88a18182a1a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:47.445381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc4603-b3b9-4e42-b343-88a18182a1a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:47.445684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc4603-b3b9-4e42-b343-88a18182a1a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:47.445963 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc4603-b3b9-4e42-b343-88a18182a1a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:47.503708 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc4603-b3b9-4e42-b343-88a18182a1a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:47.505770 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc4603-b3b9-4e42-b343-88a18182a1a1 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:47.514664 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 990/1953] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:50.999688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a53cfe-043d-4aa3-9e17-c1a36068b742 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:51.001587 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-72a53cfe-043d-4aa3-9e17-c1a36068b742 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:51.037482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:51.076102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a53cfe-043d-4aa3-9e17-c1a36068b742 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:51.081903 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:51.082597 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:51.082879 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:51.083157 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] PATH_INFO: `/v3/users/cbe55ab08dc34cd8a97af5dfe8b1be45` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:51.084330 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbe55ab08dc34cd8a97af5dfe8b1be45)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:51.086615 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:51.276876 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbe55ab08dc34cd8a97af5dfe8b1be45'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:51.277556 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbe55ab08dc34cd8a97af5dfe8b1be45'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:51.284985 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbe55ab08dc34cd8a97af5dfe8b1be45'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:51.285646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbe55ab08dc34cd8a97af5dfe8b1be45'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:51.353814 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbe55ab08dc34cd8a97af5dfe8b1be45'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:51.374538 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 964/1954] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:50 2022] DELETE /identity/v3/users/cbe55ab08dc34cd8a97af5dfe8b1be45 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:51.379020 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca1964-4528-4bdb-b951-5d48a0b33d65 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:51.379497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca1964-4528-4bdb-b951-5d48a0b33d65 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:51.379591 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca1964-4528-4bdb-b951-5d48a0b33d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:51.399055 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-89ca1964-4528-4bdb-b951-5d48a0b33d65 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:51.400245 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-89ca1964-4528-4bdb-b951-5d48a0b33d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:51.461414 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 991/1955] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:51.471132 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:51.472643 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:51.529763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6af0b-3367-4211-8494-f382494587c1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:51.533450 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3843d-a6b2-4295-97bc-d94e2cca2b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:51.534079 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3843d-a6b2-4295-97bc-d94e2cca2b6a service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:51.534414 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3843d-a6b2-4295-97bc-d94e2cca2b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:51.534693 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3843d-a6b2-4295-97bc-d94e2cca2b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:51.544637 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3843d-a6b2-4295-97bc-d94e2cca2b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:51.546146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3843d-a6b2-4295-97bc-d94e2cca2b6a service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:51.556585 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 965/1956] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:51.725762 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca1964-4528-4bdb-b951-5d48a0b33d65 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:51.727203 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-89ca1964-4528-4bdb-b951-5d48a0b33d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:51.739477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca1964-4528-4bdb-b951-5d48a0b33d65 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:51.744837 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:51.745523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:51.746126 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:51.746126 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] PATH_INFO: `/v3/users/40b0cd94bb1d4837baede8ab5532c593` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:51.747276 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b0cd94bb1d4837baede8ab5532c593)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:51.749251 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:51.751105 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:51.940716 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b0cd94bb1d4837baede8ab5532c593'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:51.948688 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b0cd94bb1d4837baede8ab5532c593'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:51.949379 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b0cd94bb1d4837baede8ab5532c593'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:51.952732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b0cd94bb1d4837baede8ab5532c593'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:51.996517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3843d-a6b2-4295-97bc-d94e2cca2b6a service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:51.997997 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5ad3843d-a6b2-4295-97bc-d94e2cca2b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:52.012046 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3843d-a6b2-4295-97bc-d94e2cca2b6a service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:52.017143 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-112155af-ea8a-49c8-a978-4d30c3a5806a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b0cd94bb1d4837baede8ab5532c593'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:52.017425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:52.018559 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.018688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.018893 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] PATH_INFO: `/v3/projects/d707e93c318640689ecb79078fbd18a1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.020529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d707e93c318640689ecb79078fbd18a1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:52.025126 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:52.039975 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 992/1957] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:51 2022] DELETE /identity/v3/users/40b0cd94bb1d4837baede8ab5532c593 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:52.044473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20778be8-6661-4e7a-ad61-c64e80202308 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.044763 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20778be8-6661-4e7a-ad61-c64e80202308 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.045041 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20778be8-6661-4e7a-ad61-c64e80202308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.065780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-20778be8-6661-4e7a-ad61-c64e80202308 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:52.067372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-20778be8-6661-4e7a-ad61-c64e80202308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:52.123703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:52.126756 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd707e93c318640689ecb79078fbd18a1', 'request_id': 'req-9636e5f0-5d8e-48e9-93b6-df18d5533df1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:52.128014 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 993/1958] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:52.135656 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd707e93c318640689ecb79078fbd18a1', 'request_id': 'req-9636e5f0-5d8e-48e9-93b6-df18d5533df1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:52.138771 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20778be8-6661-4e7a-ad61-c64e80202308 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:52.140141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-20778be8-6661-4e7a-ad61-c64e80202308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:52.199610 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd707e93c318640689ecb79078fbd18a1', 'request_id': 'req-9636e5f0-5d8e-48e9-93b6-df18d5533df1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:52.201371 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20778be8-6661-4e7a-ad61-c64e80202308 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:52.225300 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] The token cache is being invalidate because project d707e93c318640689ecb79078fbd18a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:52.226198 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:52.226565 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9636e5f0-5d8e-48e9-93b6-df18d5533df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:52.228318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 966/1959] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:51 2022] DELETE /identity/v3/projects/d707e93c318640689ecb79078fbd18a1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:52.235366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea6463-c32f-42af-869b-ec17d33b9584 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.235645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:52.235764 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea6463-c32f-42af-869b-ec17d33b9584 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.235990 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea6463-c32f-42af-869b-ec17d33b9584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.254374 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4a2b7-4cb9-4b7a-9729-4a8f9bda1518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:52.255113 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4a2b7-4cb9-4b7a-9729-4a8f9bda1518 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.255388 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4a2b7-4cb9-4b7a-9729-4a8f9bda1518 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.255657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4a2b7-4cb9-4b7a-9729-4a8f9bda1518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.256878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-12ea6463-c32f-42af-869b-ec17d33b9584 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:52.258033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-68f4a2b7-4cb9-4b7a-9729-4a8f9bda1518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:52.260306 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-12ea6463-c32f-42af-869b-ec17d33b9584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:52.319613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 967/1960] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:52.322206 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4a2b7-4cb9-4b7a-9729-4a8f9bda1518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:52.323714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4a2b7-4cb9-4b7a-9729-4a8f9bda1518 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:52.327439 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190e38f-a112-4370-b224-0397188cda2c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.327836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190e38f-a112-4370-b224-0397188cda2c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.328010 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190e38f-a112-4370-b224-0397188cda2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.334439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 994/1961] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:05:52.349151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8190e38f-a112-4370-b224-0397188cda2c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:52.350527 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8190e38f-a112-4370-b224-0397188cda2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:52.361244 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 968/1962] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:52.362993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4a2b7-4cb9-4b7a-9729-4a8f9bda1518 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:52.373052 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4a2b7-4cb9-4b7a-9729-4a8f9bda1518 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:52.377839 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca243571-66d8-47f0-bfdd-f2bf629b18c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:52.378403 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca243571-66d8-47f0-bfdd-f2bf629b18c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.378685 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca243571-66d8-47f0-bfdd-f2bf629b18c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.378959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca243571-66d8-47f0-bfdd-f2bf629b18c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.379633 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca243571-66d8-47f0-bfdd-f2bf629b18c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:52.380181 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca243571-66d8-47f0-bfdd-f2bf629b18c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:52.382115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca243571-66d8-47f0-bfdd-f2bf629b18c7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:52.394622 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 995/1963] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:05:52.399774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190e38f-a112-4370-b224-0397188cda2c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:52.409214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190e38f-a112-4370-b224-0397188cda2c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:52.414107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f407949-c43f-44c6-961e-aa39a7946e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:52.414722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f407949-c43f-44c6-961e-aa39a7946e36 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.415007 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f407949-c43f-44c6-961e-aa39a7946e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.415284 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f407949-c43f-44c6-961e-aa39a7946e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.415933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f407949-c43f-44c6-961e-aa39a7946e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:52.417949 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f407949-c43f-44c6-961e-aa39a7946e36 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:52.554243 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 969/1964] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:52 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:52.560132 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca243571-66d8-47f0-bfdd-f2bf629b18c7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:52.571736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca243571-66d8-47f0-bfdd-f2bf629b18c7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:52.631812 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ea404-8e0e-4603-a812-46374d142941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:52.632373 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ea404-8e0e-4603-a812-46374d142941 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.632657 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ea404-8e0e-4603-a812-46374d142941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.632933 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ea404-8e0e-4603-a812-46374d142941 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.633614 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ea404-8e0e-4603-a812-46374d142941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:52.635584 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ea404-8e0e-4603-a812-46374d142941 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:52.708328 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 996/1965] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:52 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:52.713209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f407949-c43f-44c6-961e-aa39a7946e36 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:52.723774 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f407949-c43f-44c6-961e-aa39a7946e36 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:52.729107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b51ff7-6a51-41f8-a11b-3da6e9be3df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:52.729751 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b51ff7-6a51-41f8-a11b-3da6e9be3df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.730035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b51ff7-6a51-41f8-a11b-3da6e9be3df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.730310 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b51ff7-6a51-41f8-a11b-3da6e9be3df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.730978 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b51ff7-6a51-41f8-a11b-3da6e9be3df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:52.732573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b51ff7-6a51-41f8-a11b-3da6e9be3df3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:52.742680 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 970/1966] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:05:52.746652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ea404-8e0e-4603-a812-46374d142941 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:52.756735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ea404-8e0e-4603-a812-46374d142941 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:52.761619 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9cdc5-5325-4193-9d38-ab96ac0d33b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:52.762189 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9cdc5-5325-4193-9d38-ab96ac0d33b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.762674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9cdc5-5325-4193-9d38-ab96ac0d33b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.762921 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9cdc5-5325-4193-9d38-ab96ac0d33b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.763392 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9cdc5-5325-4193-9d38-ab96ac0d33b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:52.765088 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9cdc5-5325-4193-9d38-ab96ac0d33b9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:52.768052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b51ff7-6a51-41f8-a11b-3da6e9be3df3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:52.770259 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-42b51ff7-6a51-41f8-a11b-3da6e9be3df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:52.778959 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 997/1967] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:52.783284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9cdc5-5325-4193-9d38-ab96ac0d33b9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:52.792999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b51ff7-6a51-41f8-a11b-3da6e9be3df3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:52.796462 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9cdc5-5325-4193-9d38-ab96ac0d33b9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:52.798670 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:52.799399 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.799603 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.799880 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] PATH_INFO: `/v3/projects/3701ad3848cd4600b76c41c59fa17401` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.800997 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3701ad3848cd4600b76c41c59fa17401)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:52.802784 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a6a82-823c-4c82-a1a4-8e989a7c0992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:52.803416 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:52.803838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a6a82-823c-4c82-a1a4-8e989a7c0992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.804191 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a6a82-823c-4c82-a1a4-8e989a7c0992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.804532 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a6a82-823c-4c82-a1a4-8e989a7c0992 admin admin] PATH_INFO: `/v3/projects/4bed17dcec874bd2838228cea29aa2c2/users/99516644a9214119bd4bcb3319bfa4e8/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.820129 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a6a82-823c-4c82-a1a4-8e989a7c0992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bed17dcec874bd2838228cea29aa2c2, user_id=99516644a9214119bd4bcb3319bfa4e8, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:52.822876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a6a82-823c-4c82-a1a4-8e989a7c0992 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:52.892093 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 998/1968] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:52 2022] PUT /identity/v3/projects/4bed17dcec874bd2838228cea29aa2c2/users/99516644a9214119bd4bcb3319bfa4e8/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:52.897713 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d524a65-eba6-45a9-8ae8-2fced5f0ed84 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:52.898001 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d524a65-eba6-45a9-8ae8-2fced5f0ed84 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:52.898293 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d524a65-eba6-45a9-8ae8-2fced5f0ed84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:52.920054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8d524a65-eba6-45a9-8ae8-2fced5f0ed84 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:52.921315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8d524a65-eba6-45a9-8ae8-2fced5f0ed84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:52.924606 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3701ad3848cd4600b76c41c59fa17401', 'request_id': 'req-7b502132-1a65-4844-8ac2-3f8fd8445cee'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:52.933461 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3701ad3848cd4600b76c41c59fa17401', 'request_id': 'req-7b502132-1a65-4844-8ac2-3f8fd8445cee'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:52.946391 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:52.986227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 999/1969] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:52.996734 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d524a65-eba6-45a9-8ae8-2fced5f0ed84 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:52.998196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8d524a65-eba6-45a9-8ae8-2fced5f0ed84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:53.000171 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3701ad3848cd4600b76c41c59fa17401', 'request_id': 'req-7b502132-1a65-4844-8ac2-3f8fd8445cee'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:53.015195 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] The token cache is being invalidate because project 3701ad3848cd4600b76c41c59fa17401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:53.015979 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:53.016146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:53.018055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 971/1970] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:52 2022] DELETE /identity/v3/projects/3701ad3848cd4600b76c41c59fa17401 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:05:53.022340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:53.023827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:53.041273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b502132-1a65-4844-8ac2-3f8fd8445cee admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:53.043683 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:53.044343 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:53.044618 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:53.044900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] PATH_INFO: `/v3/users/a2a0870ce7b343caaf36fdcc597712aa` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:53.046084 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2a0870ce7b343caaf36fdcc597712aa)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:53.048102 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:53.063402 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:53.070335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d524a65-eba6-45a9-8ae8-2fced5f0ed84 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:53.074228 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9376ba3-584a-4387-b585-d9da5ee80797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:53.074856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9376ba3-584a-4387-b585-d9da5ee80797 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:53.075114 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9376ba3-584a-4387-b585-d9da5ee80797 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:53.075386 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9376ba3-584a-4387-b585-d9da5ee80797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:53.077243 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a9376ba3-584a-4387-b585-d9da5ee80797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:53.089439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9376ba3-584a-4387-b585-d9da5ee80797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:53.090973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9376ba3-584a-4387-b585-d9da5ee80797 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:53.100725 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1000/1971] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:05:53.261164 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2a0870ce7b343caaf36fdcc597712aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:53.261981 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2a0870ce7b343caaf36fdcc597712aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:53.270364 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2a0870ce7b343caaf36fdcc597712aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:53.271154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2a0870ce7b343caaf36fdcc597712aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:53.365483 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a0734e32-a00f-4a78-b9cc-4f2c68d0bafa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2a0870ce7b343caaf36fdcc597712aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:53.385384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 972/1972] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:05:53 2022] DELETE /identity/v3/users/a2a0870ce7b343caaf36fdcc597712aa => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:53.816542 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9376ba3-584a-4387-b585-d9da5ee80797 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:53.827233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9376ba3-584a-4387-b585-d9da5ee80797 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:53.893754 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:53.894406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:53.894675 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:53.894951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] PATH_INFO: `/v3/projects/a0d1522c804448ffb9b74c9d26013b53` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:53.896070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d1522c804448ffb9b74c9d26013b53)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:53.898110 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:54.030290 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d1522c804448ffb9b74c9d26013b53', 'request_id': 'req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:54.097230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d1522c804448ffb9b74c9d26013b53', 'request_id': 'req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:54.105857 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d1522c804448ffb9b74c9d26013b53', 'request_id': 'req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:54.114050 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:54.119300 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] The token cache is being invalidate because project a0d1522c804448ffb9b74c9d26013b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:05:54.119594 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:54.120253 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6335037-2d2b-4f40-ab2f-6d5f2d76bf66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:05:54.121809 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1001/1973] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:05:53 2022] DELETE /identity/v3/projects/a0d1522c804448ffb9b74c9d26013b53 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:54.126868 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3523-557e-4e09-94c2-9f550fbebf6f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:54.127366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3523-557e-4e09-94c2-9f550fbebf6f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:54.127536 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3523-557e-4e09-94c2-9f550fbebf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:54.148561 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8fae3523-557e-4e09-94c2-9f550fbebf6f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:54.149924 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8fae3523-557e-4e09-94c2-9f550fbebf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:54.213581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 973/1974] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:05:54.222848 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3efe8c-d641-4a14-8660-aaa01113b268 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:54.223002 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3efe8c-d641-4a14-8660-aaa01113b268 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:54.223265 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3efe8c-d641-4a14-8660-aaa01113b268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:54.247988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0f3efe8c-d641-4a14-8660-aaa01113b268 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:54.249274 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0f3efe8c-d641-4a14-8660-aaa01113b268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:54.274823 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1002/1975] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:54.279245 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae3523-557e-4e09-94c2-9f550fbebf6f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:54.290788 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae3523-557e-4e09-94c2-9f550fbebf6f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:54.296780 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491510cf-6ec6-4d17-9301-5cc3e41c313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:54.297581 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491510cf-6ec6-4d17-9301-5cc3e41c313e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:54.298098 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491510cf-6ec6-4d17-9301-5cc3e41c313e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:54.298322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491510cf-6ec6-4d17-9301-5cc3e41c313e admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:54.299103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491510cf-6ec6-4d17-9301-5cc3e41c313e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:05:54.299491 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491510cf-6ec6-4d17-9301-5cc3e41c313e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:54.302592 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491510cf-6ec6-4d17-9301-5cc3e41c313e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:54.316960 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 974/1976] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:05:54.322031 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3efe8c-d641-4a14-8660-aaa01113b268 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:54.336348 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3efe8c-d641-4a14-8660-aaa01113b268 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:54.349799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bd95c-c802-41d7-943b-e15f9955c53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:54.351533 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bd95c-c802-41d7-943b-e15f9955c53c admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:54.351925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bd95c-c802-41d7-943b-e15f9955c53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:54.352198 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bd95c-c802-41d7-943b-e15f9955c53c admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:54.352852 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bd95c-c802-41d7-943b-e15f9955c53c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:54.355101 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bd95c-c802-41d7-943b-e15f9955c53c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:54.485999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1003/1977] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:54 2022] POST /identity/v3/projects => generated 409 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:54.487179 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491510cf-6ec6-4d17-9301-5cc3e41c313e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:54.499327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491510cf-6ec6-4d17-9301-5cc3e41c313e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:54.580764 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700ac56-1055-485d-acaa-67a21d3231c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:54.581883 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700ac56-1055-485d-acaa-67a21d3231c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:54.582289 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700ac56-1055-485d-acaa-67a21d3231c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:54.583169 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700ac56-1055-485d-acaa-67a21d3231c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:54.583973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700ac56-1055-485d-acaa-67a21d3231c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:54.587198 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700ac56-1055-485d-acaa-67a21d3231c3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:54.663878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 975/1978] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:54 2022] POST /identity/v3/users => generated 305 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:54.668026 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bd95c-c802-41d7-943b-e15f9955c53c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:54.679169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bd95c-c802-41d7-943b-e15f9955c53c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:54.684273 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8ef42-d82d-4311-a15d-44416adfe3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:54.684886 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8ef42-d82d-4311-a15d-44416adfe3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:54.685681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8ef42-d82d-4311-a15d-44416adfe3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:54.692270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8ef42-d82d-4311-a15d-44416adfe3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:54.692270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8ef42-d82d-4311-a15d-44416adfe3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:54.692270 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8ef42-d82d-4311-a15d-44416adfe3f3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:54.702569 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1004/1979] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:54.704098 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700ac56-1055-485d-acaa-67a21d3231c3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:54.715250 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700ac56-1055-485d-acaa-67a21d3231c3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:54.720082 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a834d-35db-49ac-abfc-b2286780ea5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:54.720690 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a834d-35db-49ac-abfc-b2286780ea5e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:54.720972 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a834d-35db-49ac-abfc-b2286780ea5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:54.721280 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a834d-35db-49ac-abfc-b2286780ea5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:54.721924 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a834d-35db-49ac-abfc-b2286780ea5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:54.723534 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a834d-35db-49ac-abfc-b2286780ea5e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:54.724917 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:54.735214 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 976/1980] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:54.739151 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8ef42-d82d-4311-a15d-44416adfe3f3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:54.749457 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8ef42-d82d-4311-a15d-44416adfe3f3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:54.754262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71721d5-3fd0-4e43-9924-819b8dd14c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:54.755130 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71721d5-3fd0-4e43-9924-819b8dd14c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:54.755474 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71721d5-3fd0-4e43-9924-819b8dd14c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:54.755816 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71721d5-3fd0-4e43-9924-819b8dd14c4a admin admin] PATH_INFO: `/v3/projects/1150a06bfcd241f68d41b11e205103f4/users/7332b2d34b09401fb0e37c3a19f1833a/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:54.784061 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71721d5-3fd0-4e43-9924-819b8dd14c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1150a06bfcd241f68d41b11e205103f4, user_id=7332b2d34b09401fb0e37c3a19f1833a, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:54.787038 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71721d5-3fd0-4e43-9924-819b8dd14c4a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:54.863484 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1005/1981] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:54 2022] PUT /identity/v3/projects/1150a06bfcd241f68d41b11e205103f4/users/7332b2d34b09401fb0e37c3a19f1833a/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:54.876061 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2f63e-a9ac-41be-894f-cb32c6f76046 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:54.876368 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2f63e-a9ac-41be-894f-cb32c6f76046 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:54.876653 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2f63e-a9ac-41be-894f-cb32c6f76046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:54.928132 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-68d2f63e-a9ac-41be-894f-cb32c6f76046 None None] MFA Rules not processed for user `7332b2d34b09401fb0e37c3a19f1833a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:54.932154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-68d2f63e-a9ac-41be-894f-cb32c6f76046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:55.021251 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 977/1982] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:54 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:55.025989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71721d5-3fd0-4e43-9924-819b8dd14c4a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:55.091012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71721d5-3fd0-4e43-9924-819b8dd14c4a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:55.095643 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b963d7-2b7d-4ac1-b2ce-b5cc74a370d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:55.096230 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b963d7-2b7d-4ac1-b2ce-b5cc74a370d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:55.096508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b963d7-2b7d-4ac1-b2ce-b5cc74a370d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:55.096780 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b963d7-2b7d-4ac1-b2ce-b5cc74a370d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:55.097442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b963d7-2b7d-4ac1-b2ce-b5cc74a370d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:55.099354 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b963d7-2b7d-4ac1-b2ce-b5cc74a370d3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:55.196530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1006/1983] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:55 2022] POST /identity/v3/projects => generated 407 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:55.197873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2f63e-a9ac-41be-894f-cb32c6f76046 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:55.234591 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2f63e-a9ac-41be-894f-cb32c6f76046 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:55.412278 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e6be9-af30-4852-895f-e4054cdad605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:55.413331 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e6be9-af30-4852-895f-e4054cdad605 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:55.413620 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e6be9-af30-4852-895f-e4054cdad605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:55.413900 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e6be9-af30-4852-895f-e4054cdad605 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:55.414645 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e6be9-af30-4852-895f-e4054cdad605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:55.416585 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e6be9-af30-4852-895f-e4054cdad605 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:55.556261 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 978/1984] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:55 2022] POST /identity/v3/users => generated 310 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:05:55.557469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b963d7-2b7d-4ac1-b2ce-b5cc74a370d3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:55.585337 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b963d7-2b7d-4ac1-b2ce-b5cc74a370d3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:55.596196 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a4d0b-faea-41be-993b-9aa3fd86b742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:55.597043 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a4d0b-faea-41be-993b-9aa3fd86b742 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:55.599835 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a4d0b-faea-41be-993b-9aa3fd86b742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:55.600758 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a4d0b-faea-41be-993b-9aa3fd86b742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:55.602492 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a4d0b-faea-41be-993b-9aa3fd86b742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:55.605398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a4d0b-faea-41be-993b-9aa3fd86b742 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:55.619856 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1007/1985] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:55.624154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e6be9-af30-4852-895f-e4054cdad605 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:55.635878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e6be9-af30-4852-895f-e4054cdad605 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:55.640179 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9ca9a-c27a-4a78-ab0f-ddb59c4bbc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:55.640855 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9ca9a-c27a-4a78-ab0f-ddb59c4bbc11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:55.641142 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9ca9a-c27a-4a78-ab0f-ddb59c4bbc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:55.641511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9ca9a-c27a-4a78-ab0f-ddb59c4bbc11 admin admin] PATH_INFO: `/v3/projects/d8948198b4b84cfdb26961dee2fd69cf/users/5febc7ec8a03459cbc4ff693b2b0024e/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:55.667133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9ca9a-c27a-4a78-ab0f-ddb59c4bbc11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8948198b4b84cfdb26961dee2fd69cf, user_id=5febc7ec8a03459cbc4ff693b2b0024e, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:55.671068 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9ca9a-c27a-4a78-ab0f-ddb59c4bbc11 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:55.778927 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 979/1986] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:55 2022] PUT /identity/v3/projects/d8948198b4b84cfdb26961dee2fd69cf/users/5febc7ec8a03459cbc4ff693b2b0024e/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:55.782760 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a4d0b-faea-41be-993b-9aa3fd86b742 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:55.795385 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a4d0b-faea-41be-993b-9aa3fd86b742 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:55.902687 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02853f-3956-4c66-bf5b-5dfe77beabac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:55.904876 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02853f-3956-4c66-bf5b-5dfe77beabac admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:55.909364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02853f-3956-4c66-bf5b-5dfe77beabac admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:55.909792 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02853f-3956-4c66-bf5b-5dfe77beabac admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:55.910808 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02853f-3956-4c66-bf5b-5dfe77beabac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:55.914859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02853f-3956-4c66-bf5b-5dfe77beabac admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:55.936424 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1008/1987] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:55.941505 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9ca9a-c27a-4a78-ab0f-ddb59c4bbc11 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:55.956151 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9ca9a-c27a-4a78-ab0f-ddb59c4bbc11 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:55.961946 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a8bd3-c646-4f5f-a7f2-e085945b6c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:55.962809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a8bd3-c646-4f5f-a7f2-e085945b6c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:55.963097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a8bd3-c646-4f5f-a7f2-e085945b6c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:55.963379 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a8bd3-c646-4f5f-a7f2-e085945b6c93 admin admin] PATH_INFO: `/v3/projects/d8948198b4b84cfdb26961dee2fd69cf/users/5febc7ec8a03459cbc4ff693b2b0024e/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:55.966340 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a8bd3-c646-4f5f-a7f2-e085945b6c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8948198b4b84cfdb26961dee2fd69cf, user_id=5febc7ec8a03459cbc4ff693b2b0024e, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:55.969326 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a8bd3-c646-4f5f-a7f2-e085945b6c93 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:56.002078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5287559-6a79-4e66-b530-bd1575fb0ed0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:56.002406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5287559-6a79-4e66-b530-bd1575fb0ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:56.003575 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5287559-6a79-4e66-b530-bd1575fb0ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:56.008211 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 980/1988] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:55 2022] PUT /identity/v3/projects/d8948198b4b84cfdb26961dee2fd69cf/users/5febc7ec8a03459cbc4ff693b2b0024e/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:05:56.020552 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:56.040294 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b5287559-6a79-4e66-b530-bd1575fb0ed0 None None] MFA Rules not processed for user `5febc7ec8a03459cbc4ff693b2b0024e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:56.042042 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b5287559-6a79-4e66-b530-bd1575fb0ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:56.139001 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1009/1989] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:56 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:56.149964 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a8bd3-c646-4f5f-a7f2-e085945b6c93 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:56.151711 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-386a8bd3-c646-4f5f-a7f2-e085945b6c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:56.215867 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:05:56.234807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a8bd3-c646-4f5f-a7f2-e085945b6c93 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:56.238536 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8ec47-0903-4a6d-bb1e-a76ea41706a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:56.239153 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ec47-0903-4a6d-bb1e-a76ea41706a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:56.239431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ec47-0903-4a6d-bb1e-a76ea41706a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:56.239703 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ec47-0903-4a6d-bb1e-a76ea41706a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:56.249682 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8ec47-0903-4a6d-bb1e-a76ea41706a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:56.254763 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8ec47-0903-4a6d-bb1e-a76ea41706a1 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:56.262174 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 981/1990] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:56 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:56.766415 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5287559-6a79-4e66-b530-bd1575fb0ed0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:56.776863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5287559-6a79-4e66-b530-bd1575fb0ed0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:56.780658 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fec4f8-7558-4d4a-b732-cf33fc969310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:56.781364 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fec4f8-7558-4d4a-b732-cf33fc969310 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:56.781651 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fec4f8-7558-4d4a-b732-cf33fc969310 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:56.781927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fec4f8-7558-4d4a-b732-cf33fc969310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:56.870493 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fec4f8-7558-4d4a-b732-cf33fc969310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:56.872068 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fec4f8-7558-4d4a-b732-cf33fc969310 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:56.883242 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1010/1991] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:05:56 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:05:59.189944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22a1f0-d409-4b96-aa57-edcbfd049f5f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:59.190406 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22a1f0-d409-4b96-aa57-edcbfd049f5f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:59.190508 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22a1f0-d409-4b96-aa57-edcbfd049f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:59.228356 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7e22a1f0-d409-4b96-aa57-edcbfd049f5f None None] MFA Rules not processed for user `99516644a9214119bd4bcb3319bfa4e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:05:59.230342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7e22a1f0-d409-4b96-aa57-edcbfd049f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:59.338024 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 982/1992] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:05:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:05:59.339204 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fec4f8-7558-4d4a-b732-cf33fc969310 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:59.340786 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-57fec4f8-7558-4d4a-b732-cf33fc969310 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:05:59.448184 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fec4f8-7558-4d4a-b732-cf33fc969310 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:59.461148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a5ce-6e76-45d7-9214-bbe00bb49e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:59.462260 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a5ce-6e76-45d7-9214-bbe00bb49e67 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:59.463305 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a5ce-6e76-45d7-9214-bbe00bb49e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:59.463884 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a5ce-6e76-45d7-9214-bbe00bb49e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:59.464798 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296a5ce-6e76-45d7-9214-bbe00bb49e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:59.467520 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296a5ce-6e76-45d7-9214-bbe00bb49e67 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:59.601383 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1011/1993] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:05:59 2022] POST /identity/v3/projects => generated 379 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:59.603568 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22a1f0-d409-4b96-aa57-edcbfd049f5f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:59.625878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22a1f0-d409-4b96-aa57-edcbfd049f5f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:59.711910 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1715ac-add1-4b77-a253-fe358761d08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:59.712417 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1715ac-add1-4b77-a253-fe358761d08b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:59.712654 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1715ac-add1-4b77-a253-fe358761d08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:59.712928 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1715ac-add1-4b77-a253-fe358761d08b admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:59.713632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1715ac-add1-4b77-a253-fe358761d08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:59.715613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1715ac-add1-4b77-a253-fe358761d08b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:59.839999 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 983/1994] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:05:59 2022] POST /identity/v3/users => generated 296 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:05:59.841336 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a5ce-6e76-45d7-9214-bbe00bb49e67 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:59.852677 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a5ce-6e76-45d7-9214-bbe00bb49e67 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:59.859307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb3ed7-e858-4917-a2e4-c04cd5197d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:59.859953 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb3ed7-e858-4917-a2e4-c04cd5197d69 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:59.860235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb3ed7-e858-4917-a2e4-c04cd5197d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:59.860536 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb3ed7-e858-4917-a2e4-c04cd5197d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:59.861438 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb3ed7-e858-4917-a2e4-c04cd5197d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:59.863116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb3ed7-e858-4917-a2e4-c04cd5197d69 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:05:59.878578 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1012/1995] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:05:59.879789 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1715ac-add1-4b77-a253-fe358761d08b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:05:59.891545 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1715ac-add1-4b77-a253-fe358761d08b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:05:59.898839 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f013c-67fc-4683-bb58-d30c5f2fd793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:05:59.900178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f013c-67fc-4683-bb58-d30c5f2fd793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:05:59.900298 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f013c-67fc-4683-bb58-d30c5f2fd793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:05:59.900701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f013c-67fc-4683-bb58-d30c5f2fd793 admin admin] PATH_INFO: `/v3/projects/f7046f6d5b63472c81418a7e375ebc2c/users/770ca81984244d1898c01315a491442a/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:05:59.948312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f013c-67fc-4683-bb58-d30c5f2fd793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7046f6d5b63472c81418a7e375ebc2c, user_id=770ca81984244d1898c01315a491442a, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:05:59.952925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f013c-67fc-4683-bb58-d30c5f2fd793 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:00.073925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 984/1996] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:05:59 2022] PUT /identity/v3/projects/f7046f6d5b63472c81418a7e375ebc2c/users/770ca81984244d1898c01315a491442a/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:06:00.078221 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb3ed7-e858-4917-a2e4-c04cd5197d69 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:00.085469 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:00.089482 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb3ed7-e858-4917-a2e4-c04cd5197d69 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:00.159882 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18900db-c057-4e36-abe6-650f81aa1a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:00.160501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18900db-c057-4e36-abe6-650f81aa1a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:00.160785 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18900db-c057-4e36-abe6-650f81aa1a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:00.161330 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18900db-c057-4e36-abe6-650f81aa1a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:00.161738 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18900db-c057-4e36-abe6-650f81aa1a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:00.164049 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18900db-c057-4e36-abe6-650f81aa1a24 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:00.174836 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1013/1997] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:06:00.179516 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f013c-67fc-4683-bb58-d30c5f2fd793 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:00.190532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f013c-67fc-4683-bb58-d30c5f2fd793 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:00.195596 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b570-5f1e-4e5f-8d3f-3b7874af150f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:00.196274 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b570-5f1e-4e5f-8d3f-3b7874af150f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:00.196561 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b570-5f1e-4e5f-8d3f-3b7874af150f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:00.196843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b570-5f1e-4e5f-8d3f-3b7874af150f admin admin] PATH_INFO: `/v3/projects/f7046f6d5b63472c81418a7e375ebc2c/users/770ca81984244d1898c01315a491442a/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:00.199118 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5b570-5f1e-4e5f-8d3f-3b7874af150f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7046f6d5b63472c81418a7e375ebc2c, user_id=770ca81984244d1898c01315a491442a, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:00.202017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5b570-5f1e-4e5f-8d3f-3b7874af150f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:00.227318 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 985/1998] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:06:00 2022] PUT /identity/v3/projects/f7046f6d5b63472c81418a7e375ebc2c/users/770ca81984244d1898c01315a491442a/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:02.119872 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918872e0-bfc2-42ce-a878-e0cf54639372 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:02.120439 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918872e0-bfc2-42ce-a878-e0cf54639372 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:02.120534 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918872e0-bfc2-42ce-a878-e0cf54639372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:02.159456 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-918872e0-bfc2-42ce-a878-e0cf54639372 None None] MFA Rules not processed for user `c36b6b9fbf194346bfc0a7e277c72eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:02.161046 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-918872e0-bfc2-42ce-a878-e0cf54639372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:02.235045 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1014/1999] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:06:07.106223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf64a75-9155-4e31-823b-24590f88f7c0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:07.106760 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf64a75-9155-4e31-823b-24590f88f7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:07.106760 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf64a75-9155-4e31-823b-24590f88f7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:07.140646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-baf64a75-9155-4e31-823b-24590f88f7c0 None None] MFA Rules not processed for user `770ca81984244d1898c01315a491442a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:07.141925 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-baf64a75-9155-4e31-823b-24590f88f7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:07.154222 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:07.249431 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 986/2000] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:06:07.266973 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918872e0-bfc2-42ce-a878-e0cf54639372 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:07.268480 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-918872e0-bfc2-42ce-a878-e0cf54639372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:07.386752 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918872e0-bfc2-42ce-a878-e0cf54639372 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:07.393342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1be56-f1be-4849-a629-4cc2b354c79d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:07.394175 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1be56-f1be-4849-a629-4cc2b354c79d service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:07.394459 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1be56-f1be-4849-a629-4cc2b354c79d service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:07.394736 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1be56-f1be-4849-a629-4cc2b354c79d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:07.464257 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:07.482207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1be56-f1be-4849-a629-4cc2b354c79d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:07.487302 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1be56-f1be-4849-a629-4cc2b354c79d service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:07.509289 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1015/2001] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:06:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:06:09.913041 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf64a75-9155-4e31-823b-24590f88f7c0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:10.011262 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf64a75-9155-4e31-823b-24590f88f7c0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:10.020600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:10.020600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:10.020600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:10.020600 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] PATH_INFO: `/v3/users/7332b2d34b09401fb0e37c3a19f1833a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:10.021141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7332b2d34b09401fb0e37c3a19f1833a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:10.021141 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:10.244363 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7332b2d34b09401fb0e37c3a19f1833a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:10.245178 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7332b2d34b09401fb0e37c3a19f1833a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:10.253984 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7332b2d34b09401fb0e37c3a19f1833a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:10.254730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7332b2d34b09401fb0e37c3a19f1833a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:10.320424 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7332b2d34b09401fb0e37c3a19f1833a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:10.343757 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 987/2002] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:06:09 2022] DELETE /identity/v3/users/7332b2d34b09401fb0e37c3a19f1833a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:10.348909 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83c3a8-8509-4916-8a80-7d8bbd196d33 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:10.350240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83c3a8-8509-4916-8a80-7d8bbd196d33 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:10.351322 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83c3a8-8509-4916-8a80-7d8bbd196d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:10.374169 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8f83c3a8-8509-4916-8a80-7d8bbd196d33 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:10.375408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8f83c3a8-8509-4916-8a80-7d8bbd196d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:10.453976 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1016/2003] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:06:10.464309 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:10.465827 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:10.503613 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:10.549787 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc4bb-f1be-42ca-b41a-d83980622083 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:10.562425 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01400193-82c1-40da-8946-c71a90f5aa06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:10.563520 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01400193-82c1-40da-8946-c71a90f5aa06 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:10.564202 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01400193-82c1-40da-8946-c71a90f5aa06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:10.564524 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01400193-82c1-40da-8946-c71a90f5aa06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:10.577220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01400193-82c1-40da-8946-c71a90f5aa06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:10.578951 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01400193-82c1-40da-8946-c71a90f5aa06 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:10.592213 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 988/2004] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:06:11.228563 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83c3a8-8509-4916-8a80-7d8bbd196d33 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:11.230193 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8f83c3a8-8509-4916-8a80-7d8bbd196d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:11.266083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83c3a8-8509-4916-8a80-7d8bbd196d33 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:11.286256 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:11.287609 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:11.287906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:11.288252 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] PATH_INFO: `/v3/projects/1150a06bfcd241f68d41b11e205103f4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:11.289859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1150a06bfcd241f68d41b11e205103f4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:11.291807 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:11.391588 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1150a06bfcd241f68d41b11e205103f4', 'request_id': 'req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.479497 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1150a06bfcd241f68d41b11e205103f4', 'request_id': 'req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.486434 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1150a06bfcd241f68d41b11e205103f4', 'request_id': 'req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.497589 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] The token cache is being invalidate because project 1150a06bfcd241f68d41b11e205103f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:06:11.497902 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.498596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.500167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1017/2005] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:06:11 2022] DELETE /identity/v3/projects/1150a06bfcd241f68d41b11e205103f4 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:11.504082 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01400193-82c1-40da-8946-c71a90f5aa06 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:11.514598 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01400193-82c1-40da-8946-c71a90f5aa06 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:11.599825 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:11.600550 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:11.600828 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:11.601322 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] PATH_INFO: `/v3/users/5febc7ec8a03459cbc4ff693b2b0024e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:11.615500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5febc7ec8a03459cbc4ff693b2b0024e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:11.615500 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:11.801630 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5febc7ec8a03459cbc4ff693b2b0024e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.802730 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5febc7ec8a03459cbc4ff693b2b0024e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.810184 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5febc7ec8a03459cbc4ff693b2b0024e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.810854 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5febc7ec8a03459cbc4ff693b2b0024e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.885401 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d710bc04-ee7d-408f-951d-08345c61be65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5febc7ec8a03459cbc4ff693b2b0024e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:11.906805 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 989/2006] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:06:11 2022] DELETE /identity/v3/users/5febc7ec8a03459cbc4ff693b2b0024e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:12.315306 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:12.320623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:12.342810 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaf47f-1987-4ce7-b296-89d5e5c2b6f2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:12.413474 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:12.414674 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:12.415141 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:12.415486 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] PATH_INFO: `/v3/projects/d8948198b4b84cfdb26961dee2fd69cf` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:12.416883 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8948198b4b84cfdb26961dee2fd69cf)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:12.418960 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:12.429236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:12.538066 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8948198b4b84cfdb26961dee2fd69cf', 'request_id': 'req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:12.617078 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8948198b4b84cfdb26961dee2fd69cf', 'request_id': 'req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:12.625618 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8948198b4b84cfdb26961dee2fd69cf', 'request_id': 'req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:12.637775 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] The token cache is being invalidate because project d8948198b4b84cfdb26961dee2fd69cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:06:12.638131 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:12.638794 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a53fe1c1-7a00-4dcc-90bb-d753abd9703b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:12.640596 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1018/2007] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:06:12 2022] DELETE /identity/v3/projects/d8948198b4b84cfdb26961dee2fd69cf => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:12.645770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4302-12be-4d7e-b2ec-bedda27d5b3f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:12.647003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4302-12be-4d7e-b2ec-bedda27d5b3f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:12.647003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4302-12be-4d7e-b2ec-bedda27d5b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:12.673936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4a6e4302-12be-4d7e-b2ec-bedda27d5b3f None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:12.675155 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4a6e4302-12be-4d7e-b2ec-bedda27d5b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:12.721030 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:12.736959 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 990/2008] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:06:12.746282 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340c60f-198d-4239-a752-6991e4ee4c46 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:12.746572 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340c60f-198d-4239-a752-6991e4ee4c46 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:12.746850 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340c60f-198d-4239-a752-6991e4ee4c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:12.766524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9340c60f-198d-4239-a752-6991e4ee4c46 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:12.769227 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9340c60f-198d-4239-a752-6991e4ee4c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:12.776408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1019/2009] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:06:12.781116 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4302-12be-4d7e-b2ec-bedda27d5b3f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:12.792033 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4302-12be-4d7e-b2ec-bedda27d5b3f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:12.796684 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05db46-c033-4402-8d87-3fb31b6bb170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:12.797336 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05db46-c033-4402-8d87-3fb31b6bb170 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:12.797619 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05db46-c033-4402-8d87-3fb31b6bb170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:12.797897 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05db46-c033-4402-8d87-3fb31b6bb170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:12.798673 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05db46-c033-4402-8d87-3fb31b6bb170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:06:12.800713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05db46-c033-4402-8d87-3fb31b6bb170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:12.800713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05db46-c033-4402-8d87-3fb31b6bb170 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:12.814394 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 991/2010] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:06:12.818442 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340c60f-198d-4239-a752-6991e4ee4c46 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:12.827912 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340c60f-198d-4239-a752-6991e4ee4c46 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:12.832550 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36be13-1a73-4e07-ac52-745d9ea9f3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:12.833146 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36be13-1a73-4e07-ac52-745d9ea9f3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:12.834004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36be13-1a73-4e07-ac52-745d9ea9f3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:12.834280 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36be13-1a73-4e07-ac52-745d9ea9f3c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:12.835019 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36be13-1a73-4e07-ac52-745d9ea9f3c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:12.836947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36be13-1a73-4e07-ac52-745d9ea9f3c6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:12.948187 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1020/2011] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:06:12 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:06:12.949419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05db46-c033-4402-8d87-3fb31b6bb170 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:12.961043 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05db46-c033-4402-8d87-3fb31b6bb170 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:13.032586 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1ba8c-00c8-456d-ad1a-49c4f68250bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:13.035945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1ba8c-00c8-456d-ad1a-49c4f68250bd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:13.035945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1ba8c-00c8-456d-ad1a-49c4f68250bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:13.035945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1ba8c-00c8-456d-ad1a-49c4f68250bd admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:13.035945 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1ba8c-00c8-456d-ad1a-49c4f68250bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:13.037796 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1ba8c-00c8-456d-ad1a-49c4f68250bd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:13.234566 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 992/2012] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:06:12 2022] POST /identity/v3/users => generated 288 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:06:13.238763 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36be13-1a73-4e07-ac52-745d9ea9f3c6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:13.249690 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36be13-1a73-4e07-ac52-745d9ea9f3c6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:13.254126 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6f95c-29f9-4f01-9c49-64088800eba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:13.254719 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f95c-29f9-4f01-9c49-64088800eba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:13.254999 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f95c-29f9-4f01-9c49-64088800eba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:13.255275 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f95c-29f9-4f01-9c49-64088800eba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:13.256623 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6f95c-29f9-4f01-9c49-64088800eba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:13.258255 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6f95c-29f9-4f01-9c49-64088800eba6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:13.268012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1021/2013] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:06:13.272702 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1ba8c-00c8-456d-ad1a-49c4f68250bd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:13.283809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1ba8c-00c8-456d-ad1a-49c4f68250bd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:13.288989 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2541fb6-1d12-4e3c-9cee-f55f52452316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:13.289552 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2541fb6-1d12-4e3c-9cee-f55f52452316 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:13.289835 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2541fb6-1d12-4e3c-9cee-f55f52452316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:13.290110 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2541fb6-1d12-4e3c-9cee-f55f52452316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:13.290754 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2541fb6-1d12-4e3c-9cee-f55f52452316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:13.292382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2541fb6-1d12-4e3c-9cee-f55f52452316 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:13.303838 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 993/2014] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:06:13.308084 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6f95c-29f9-4f01-9c49-64088800eba6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:13.318732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6f95c-29f9-4f01-9c49-64088800eba6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:13.323343 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e284d-d764-4c25-ae54-fb0024ab3ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:13.323991 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e284d-d764-4c25-ae54-fb0024ab3ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:13.324380 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e284d-d764-4c25-ae54-fb0024ab3ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:13.324671 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e284d-d764-4c25-ae54-fb0024ab3ed6 admin admin] PATH_INFO: `/v3/projects/4d1cdfe211234bcfbc65e485b4c49269/users/43d6403be926433fa85e7f150005da32/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:13.346519 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e284d-d764-4c25-ae54-fb0024ab3ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1cdfe211234bcfbc65e485b4c49269, user_id=43d6403be926433fa85e7f150005da32, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:13.349767 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e284d-d764-4c25-ae54-fb0024ab3ed6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:13.431031 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1022/2015] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:06:13 2022] PUT /identity/v3/projects/4d1cdfe211234bcfbc65e485b4c49269/users/43d6403be926433fa85e7f150005da32/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:13.442584 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6ed1-270e-4c25-9258-4ac544707fe1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:13.443575 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6ed1-270e-4c25-9258-4ac544707fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:13.444054 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6ed1-270e-4c25-9258-4ac544707fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:13.476015 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b90f6ed1-270e-4c25-9258-4ac544707fe1 None None] MFA Rules not processed for user `43d6403be926433fa85e7f150005da32`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:13.477261 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b90f6ed1-270e-4c25-9258-4ac544707fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:13.561522 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 994/2016] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:06:13.566177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e284d-d764-4c25-ae54-fb0024ab3ed6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:13.641906 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e284d-d764-4c25-ae54-fb0024ab3ed6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:13.650016 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a3dea-5554-49ff-8827-df15d9e8fb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:13.651356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3dea-5554-49ff-8827-df15d9e8fb4d admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:13.651356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3dea-5554-49ff-8827-df15d9e8fb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:13.651356 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3dea-5554-49ff-8827-df15d9e8fb4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:13.651814 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a3dea-5554-49ff-8827-df15d9e8fb4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:13.653772 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a3dea-5554-49ff-8827-df15d9e8fb4d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:13.744083 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1023/2017] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:06:13 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:06:13.747933 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f6ed1-270e-4c25-9258-4ac544707fe1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:13.761001 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f6ed1-270e-4c25-9258-4ac544707fe1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:13.806911 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:13.824025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da054e-442b-4704-824f-73e469ddd763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:13.824689 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da054e-442b-4704-824f-73e469ddd763 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:13.824943 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da054e-442b-4704-824f-73e469ddd763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:13.825239 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da054e-442b-4704-824f-73e469ddd763 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:13.825977 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da054e-442b-4704-824f-73e469ddd763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:13.827924 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da054e-442b-4704-824f-73e469ddd763 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:13.959516 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 995/2018] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:06:13 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:06:13.963678 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a3dea-5554-49ff-8827-df15d9e8fb4d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:13.974602 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a3dea-5554-49ff-8827-df15d9e8fb4d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:13.979105 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686ff53-184c-4280-98d6-3326b41308a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:13.979729 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686ff53-184c-4280-98d6-3326b41308a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:13.980010 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686ff53-184c-4280-98d6-3326b41308a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:13.980329 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686ff53-184c-4280-98d6-3326b41308a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:13.981004 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686ff53-184c-4280-98d6-3326b41308a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:13.982639 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686ff53-184c-4280-98d6-3326b41308a3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:13.994396 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1024/2019] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:06:14.000878 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da054e-442b-4704-824f-73e469ddd763 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:14.013580 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da054e-442b-4704-824f-73e469ddd763 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:14.020988 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417fdbe-e663-4f9b-b0be-6eeec22a62de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:14.024544 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417fdbe-e663-4f9b-b0be-6eeec22a62de admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:14.024544 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417fdbe-e663-4f9b-b0be-6eeec22a62de admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:14.024544 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417fdbe-e663-4f9b-b0be-6eeec22a62de admin admin] PATH_INFO: `/v3/projects/c1fa025ce4194bdb9d37c6097eed445c/users/2c58ee6df5854e44bc6cbfb15e69ff3f/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:14.045624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417fdbe-e663-4f9b-b0be-6eeec22a62de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fa025ce4194bdb9d37c6097eed445c, user_id=2c58ee6df5854e44bc6cbfb15e69ff3f, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:14.048696 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417fdbe-e663-4f9b-b0be-6eeec22a62de admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:14.121688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 996/2020] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:06:13 2022] PUT /identity/v3/projects/c1fa025ce4194bdb9d37c6097eed445c/users/2c58ee6df5854e44bc6cbfb15e69ff3f/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:14.126573 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686ff53-184c-4280-98d6-3326b41308a3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:14.142240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686ff53-184c-4280-98d6-3326b41308a3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:14.148089 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:14.211511 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ce09b-fe6f-41fd-8875-db5c1a52b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:14.212054 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ce09b-fe6f-41fd-8875-db5c1a52b0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:14.212334 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ce09b-fe6f-41fd-8875-db5c1a52b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:14.212612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ce09b-fe6f-41fd-8875-db5c1a52b0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:14.213318 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ce09b-fe6f-41fd-8875-db5c1a52b0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:14.215148 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ce09b-fe6f-41fd-8875-db5c1a52b0e8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:14.233863 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1025/2021] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:06:14.235053 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417fdbe-e663-4f9b-b0be-6eeec22a62de admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:14.246292 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417fdbe-e663-4f9b-b0be-6eeec22a62de admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:14.251052 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00a199-2112-4294-bfa9-f008550e7f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:14.251722 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00a199-2112-4294-bfa9-f008550e7f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:14.252006 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00a199-2112-4294-bfa9-f008550e7f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:14.252279 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00a199-2112-4294-bfa9-f008550e7f4f admin admin] PATH_INFO: `/v3/projects/c1fa025ce4194bdb9d37c6097eed445c/users/2c58ee6df5854e44bc6cbfb15e69ff3f/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:14.254249 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00a199-2112-4294-bfa9-f008550e7f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fa025ce4194bdb9d37c6097eed445c, user_id=2c58ee6df5854e44bc6cbfb15e69ff3f, role_id=d9dfb65e98dd4a1faf4a37ccf27f27b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:14.260154 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00a199-2112-4294-bfa9-f008550e7f4f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:14.270107 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 997/2022] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:06:14 2022] PUT /identity/v3/projects/c1fa025ce4194bdb9d37c6097eed445c/users/2c58ee6df5854e44bc6cbfb15e69ff3f/roles/d9dfb65e98dd4a1faf4a37ccf27f27b5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:14.282645 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f179dd-5977-42e3-bef5-ba5ba87a7c93 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:14.282967 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f179dd-5977-42e3-bef5-ba5ba87a7c93 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:14.283247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f179dd-5977-42e3-bef5-ba5ba87a7c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:14.317214 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e1f179dd-5977-42e3-bef5-ba5ba87a7c93 None None] MFA Rules not processed for user `2c58ee6df5854e44bc6cbfb15e69ff3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:14.318764 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e1f179dd-5977-42e3-bef5-ba5ba87a7c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:14.393105 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1026/2023] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:06:14.404624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00a199-2112-4294-bfa9-f008550e7f4f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:14.406460 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0e00a199-2112-4294-bfa9-f008550e7f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:14.469936 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00a199-2112-4294-bfa9-f008550e7f4f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:14.473848 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef8105-1d77-4ce3-8ac0-8a0974d87cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264bc0b947614ec2a554d627e9e2da56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:14.474819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef8105-1d77-4ce3-8ac0-8a0974d87cad service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:14.475100 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef8105-1d77-4ce3-8ac0-8a0974d87cad service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:14.475373 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef8105-1d77-4ce3-8ac0-8a0974d87cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:14.534295 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef8105-1d77-4ce3-8ac0-8a0974d87cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:14.535743 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef8105-1d77-4ce3-8ac0-8a0974d87cad service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:14.545021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 998/2024] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:06:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:06:18.812033 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f179dd-5977-42e3-bef5-ba5ba87a7c93 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:18.886796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f179dd-5977-42e3-bef5-ba5ba87a7c93 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:18.892081 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:18.892714 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:18.892995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:18.893378 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] PATH_INFO: `/v3/users/43d6403be926433fa85e7f150005da32` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:18.894521 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d6403be926433fa85e7f150005da32)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:18.896646 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:19.149073 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:19.164278 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d6403be926433fa85e7f150005da32'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:19.171686 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d6403be926433fa85e7f150005da32'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:19.172400 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d6403be926433fa85e7f150005da32'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:19.173043 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d6403be926433fa85e7f150005da32'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:19.233503 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d6403be926433fa85e7f150005da32'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:19.252247 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1027/2025] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:06:18 2022] DELETE /identity/v3/users/43d6403be926433fa85e7f150005da32 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:19.257384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fefa3f-290b-4232-a26c-b99fbef7075e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:19.257913 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fefa3f-290b-4232-a26c-b99fbef7075e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:19.258128 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fefa3f-290b-4232-a26c-b99fbef7075e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:19.279327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-53fefa3f-290b-4232-a26c-b99fbef7075e None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:19.280554 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53fefa3f-290b-4232-a26c-b99fbef7075e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:19.326713 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:19.341635 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 999/2026] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:06:19.351284 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:19.352799 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:19.413836 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b96b3-e59c-4ed3-bc09-5c61bc1123bb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:19.417312 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a9920-e50c-40f2-8851-ae28aea9cec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:19.417932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a9920-e50c-40f2-8851-ae28aea9cec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:19.418365 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a9920-e50c-40f2-8851-ae28aea9cec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:19.418651 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a9920-e50c-40f2-8851-ae28aea9cec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:19.429177 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a9920-e50c-40f2-8851-ae28aea9cec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:19.430684 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a9920-e50c-40f2-8851-ae28aea9cec3 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:19.440344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1028/2027] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:06:19.846468 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fefa3f-290b-4232-a26c-b99fbef7075e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:19.851248 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53fefa3f-290b-4232-a26c-b99fbef7075e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:19.861180 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fefa3f-290b-4232-a26c-b99fbef7075e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:19.865886 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:19.866658 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:19.866941 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:19.867217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] PATH_INFO: `/v3/projects/4d1cdfe211234bcfbc65e485b4c49269` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:19.868545 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1cdfe211234bcfbc65e485b4c49269)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:19.870380 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:19.973208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1cdfe211234bcfbc65e485b4c49269', 'request_id': 'req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:19.979732 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1cdfe211234bcfbc65e485b4c49269', 'request_id': 'req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:20.062249 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1cdfe211234bcfbc65e485b4c49269', 'request_id': 'req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:20.074342 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] The token cache is being invalidate because project 4d1cdfe211234bcfbc65e485b4c49269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:06:20.080501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:20.080501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:20.080501 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1000/2028] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:06:19 2022] DELETE /identity/v3/projects/4d1cdfe211234bcfbc65e485b4c49269 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:20.082244 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a9920-e50c-40f2-8851-ae28aea9cec3 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:20.093116 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a9920-e50c-40f2-8851-ae28aea9cec3 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:20.163993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:20.164683 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:20.165012 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:20.165335 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] PATH_INFO: `/v3/users/2c58ee6df5854e44bc6cbfb15e69ff3f` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:20.166530 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c58ee6df5854e44bc6cbfb15e69ff3f)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:20.168479 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:20.333951 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c58ee6df5854e44bc6cbfb15e69ff3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:20.338892 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:20.341894 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c58ee6df5854e44bc6cbfb15e69ff3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:20.342612 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c58ee6df5854e44bc6cbfb15e69ff3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:20.343182 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c58ee6df5854e44bc6cbfb15e69ff3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:20.421815 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86c7a733-a15d-491a-a2e6-bfe53919675c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c58ee6df5854e44bc6cbfb15e69ff3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:20.446262 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1029/2029] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:06:20 2022] DELETE /identity/v3/users/2c58ee6df5854e44bc6cbfb15e69ff3f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:20.857960 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:20.859479 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:20.872862 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff6a8b-17a3-46bb-802e-00f4cba6f8a0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:20.936070 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:20.936719 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:20.937003 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:20.937316 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] PATH_INFO: `/v3/projects/c1fa025ce4194bdb9d37c6097eed445c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:20.938480 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1fa025ce4194bdb9d37c6097eed445c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:20.940408 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:21.045729 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1fa025ce4194bdb9d37c6097eed445c', 'request_id': 'req-871ab773-5e41-4cb9-ad9f-13790240d2d2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:21.052632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1fa025ce4194bdb9d37c6097eed445c', 'request_id': 'req-871ab773-5e41-4cb9-ad9f-13790240d2d2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:21.124814 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1fa025ce4194bdb9d37c6097eed445c', 'request_id': 'req-871ab773-5e41-4cb9-ad9f-13790240d2d2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:21.137183 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] The token cache is being invalidate because project c1fa025ce4194bdb9d37c6097eed445c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:06:21.137533 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:21.138220 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-871ab773-5e41-4cb9-ad9f-13790240d2d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:06:21.139819 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1001/2030] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:06:20 2022] DELETE /identity/v3/projects/c1fa025ce4194bdb9d37c6097eed445c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:06:21.145562 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b3775-54ec-4b44-8e77-6ada13d36f67 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:21.157681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b3775-54ec-4b44-8e77-6ada13d36f67 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:21.157681 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b3775-54ec-4b44-8e77-6ada13d36f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:21.170043 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-805b3775-54ec-4b44-8e77-6ada13d36f67 None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:21.171349 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-805b3775-54ec-4b44-8e77-6ada13d36f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:21.244412 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1030/2031] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:06:21.246086 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80da89e-efe0-40e7-8ef8-5e78acd3ba3c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:21.246378 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80da89e-efe0-40e7-8ef8-5e78acd3ba3c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:21.246660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80da89e-efe0-40e7-8ef8-5e78acd3ba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:21.255444 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:21.267103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b80da89e-efe0-40e7-8ef8-5e78acd3ba3c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:21.268325 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b80da89e-efe0-40e7-8ef8-5e78acd3ba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:21.277223 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1002/2032] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:06:21.281249 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b3775-54ec-4b44-8e77-6ada13d36f67 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:21.295873 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b3775-54ec-4b44-8e77-6ada13d36f67 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:21.297133 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15579b-c6e6-43ef-aef9-3732349e7c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:21.297782 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15579b-c6e6-43ef-aef9-3732349e7c87 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:21.298067 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15579b-c6e6-43ef-aef9-3732349e7c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:21.298344 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15579b-c6e6-43ef-aef9-3732349e7c87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:21.298998 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db15579b-c6e6-43ef-aef9-3732349e7c87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:06:21.299307 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db15579b-c6e6-43ef-aef9-3732349e7c87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:21.300968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db15579b-c6e6-43ef-aef9-3732349e7c87 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:21.314952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1031/2033] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 13:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:06:21.320277 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80da89e-efe0-40e7-8ef8-5e78acd3ba3c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:21.344463 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80da89e-efe0-40e7-8ef8-5e78acd3ba3c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:21.349639 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23df1a-9346-468d-8c97-d609304a2d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:21.351291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23df1a-9346-468d-8c97-d609304a2d23 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:21.351291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23df1a-9346-468d-8c97-d609304a2d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:21.352123 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23df1a-9346-468d-8c97-d609304a2d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:21.352384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23df1a-9346-468d-8c97-d609304a2d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:21.354535 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23df1a-9346-468d-8c97-d609304a2d23 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:21.463477 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1003/2034] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 13:06:21 2022] POST /identity/v3/projects => generated 369 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:06:21.467476 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15579b-c6e6-43ef-aef9-3732349e7c87 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:21.479201 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15579b-c6e6-43ef-aef9-3732349e7c87 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:21.554167 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf867-57e2-4ec1-a27d-31062de3af4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:21.554777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf867-57e2-4ec1-a27d-31062de3af4a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:21.555123 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf867-57e2-4ec1-a27d-31062de3af4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:21.555501 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf867-57e2-4ec1-a27d-31062de3af4a admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:21.556215 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cf867-57e2-4ec1-a27d-31062de3af4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:21.558350 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cf867-57e2-4ec1-a27d-31062de3af4a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:21.660342 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1032/2035] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 13:06:21 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:06:21.661419 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23df1a-9346-468d-8c97-d609304a2d23 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:21.674755 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23df1a-9346-468d-8c97-d609304a2d23 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:21.680361 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88894841-5355-4ade-832c-5469f3b750a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:21.680973 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88894841-5355-4ade-832c-5469f3b750a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:21.681291 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88894841-5355-4ade-832c-5469f3b750a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:21.681591 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88894841-5355-4ade-832c-5469f3b750a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:21.682225 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88894841-5355-4ade-832c-5469f3b750a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:21.683905 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88894841-5355-4ade-832c-5469f3b750a0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:21.694701 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1004/2036] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:06:21.698398 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf867-57e2-4ec1-a27d-31062de3af4a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:21.708843 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf867-57e2-4ec1-a27d-31062de3af4a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:21.714609 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacbea3-5465-4273-a78d-534d5f9284ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:21.715240 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacbea3-5465-4273-a78d-534d5f9284ba admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:21.715518 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacbea3-5465-4273-a78d-534d5f9284ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:21.715793 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacbea3-5465-4273-a78d-534d5f9284ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:21.716445 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacbea3-5465-4273-a78d-534d5f9284ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:21.718090 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacbea3-5465-4273-a78d-534d5f9284ba admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:21.730528 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1033/2037] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 13:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:06:21.734650 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88894841-5355-4ade-832c-5469f3b750a0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:21.745436 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88894841-5355-4ade-832c-5469f3b750a0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:21.750035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed82a1-1fb9-47e5-9361-a4e22636b8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:21.750700 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed82a1-1fb9-47e5-9361-a4e22636b8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:21.750985 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed82a1-1fb9-47e5-9361-a4e22636b8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:21.751423 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed82a1-1fb9-47e5-9361-a4e22636b8b2 admin admin] PATH_INFO: `/v3/projects/457ba94b915d49b9b922982cfeb4535e/users/912e26aec2724786816502cfb80b67ed/roles/f1fda4fad003483dbc5843e16b6dc9d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:21.769112 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed82a1-1fb9-47e5-9361-a4e22636b8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=457ba94b915d49b9b922982cfeb4535e, user_id=912e26aec2724786816502cfb80b67ed, role_id=f1fda4fad003483dbc5843e16b6dc9d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:21.772490 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed82a1-1fb9-47e5-9361-a4e22636b8b2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:21.828884 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1005/2038] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 13:06:21 2022] PUT /identity/v3/projects/457ba94b915d49b9b922982cfeb4535e/users/912e26aec2724786816502cfb80b67ed/roles/f1fda4fad003483dbc5843e16b6dc9d2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:06:21.840304 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471027-5e05-453f-8865-4652ba0e3d2d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:21.840666 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471027-5e05-453f-8865-4652ba0e3d2d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:21.841319 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471027-5e05-453f-8865-4652ba0e3d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:21.873061 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4a471027-5e05-453f-8865-4652ba0e3d2d None None] MFA Rules not processed for user `912e26aec2724786816502cfb80b67ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:21.874917 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4a471027-5e05-453f-8865-4652ba0e3d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:21.902652 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:21.952195 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1034/2039] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:06:21.964285 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed82a1-1fb9-47e5-9361-a4e22636b8b2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:21.966228 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aeed82a1-1fb9-47e5-9361-a4e22636b8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:22.054497 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed82a1-1fb9-47e5-9361-a4e22636b8b2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:22.061387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae705d-c4e7-4da9-a1b7-5e8ed7e76e8d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc4d28cded84afba73b053d18d4aa23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:22.061387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae705d-c4e7-4da9-a1b7-5e8ed7e76e8d service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:22.061387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae705d-c4e7-4da9-a1b7-5e8ed7e76e8d service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:22.061387 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae705d-c4e7-4da9-a1b7-5e8ed7e76e8d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:22.071517 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae705d-c4e7-4da9-a1b7-5e8ed7e76e8d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:22.073051 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae705d-c4e7-4da9-a1b7-5e8ed7e76e8d service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:22.082632 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1006/2040] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:06:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:06:24.858488 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a471027-5e05-453f-8865-4652ba0e3d2d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:24.859922 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4a471027-5e05-453f-8865-4652ba0e3d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:24.917693 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a471027-5e05-453f-8865-4652ba0e3d2d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:24.921281 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:24.921886 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:24.922168 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:24.922440 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:24.924111 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:24.988301 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:24.989777 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:24.999859 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1035/2041] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:06:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:06:32.078176 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=60,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:34.995551 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=66 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:41.278836 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fcd42-24ae-48ee-89d2-fb7663275585 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:41.279337 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fcd42-24ae-48ee-89d2-fb7663275585 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:41.279454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fcd42-24ae-48ee-89d2-fb7663275585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:41.299233 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cc3fcd42-24ae-48ee-89d2-fb7663275585 None None] MFA Rules not processed for user `99516644a9214119bd4bcb3319bfa4e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:06:41.300454 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cc3fcd42-24ae-48ee-89d2-fb7663275585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:41.357824 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1007/2042] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:06:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:06:43.655796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:06:43.657236 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:43.726216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d854a8-086f-4cca-b8db-aa27c9b4c2b1 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:06:43.731986 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06584c-7290-44ab-8a1f-9215125e72fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18df1f401bb2494299eda9582c85a04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:06:43.732584 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06584c-7290-44ab-8a1f-9215125e72fa service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:06:43.732864 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06584c-7290-44ab-8a1f-9215125e72fa service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:06:43.733138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06584c-7290-44ab-8a1f-9215125e72fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:06:43.734968 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7b06584c-7290-44ab-8a1f-9215125e72fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:06:43.746257 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06584c-7290-44ab-8a1f-9215125e72fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:06:43.747930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06584c-7290-44ab-8a1f-9215125e72fa service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:06:43.757728 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1036/2043] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:06:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:06:51.350093 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=25 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:06:53.753702 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=32 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:07:10.599688 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fcd42-24ae-48ee-89d2-fb7663275585 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:07:10.660660 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fcd42-24ae-48ee-89d2-fb7663275585 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:07:10.665209 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:07:10.665865 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:10.666133 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:10.666411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] PATH_INFO: `/v3/users/912e26aec2724786816502cfb80b67ed` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:10.668304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] RBAC: Authorizing `identity:delete_user(user_id=912e26aec2724786816502cfb80b67ed)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:07:10.670273 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:07:10.874825 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '912e26aec2724786816502cfb80b67ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:10.875535 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '912e26aec2724786816502cfb80b67ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:10.882924 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '912e26aec2724786816502cfb80b67ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:10.883624 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '912e26aec2724786816502cfb80b67ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:10.969132 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '912e26aec2724786816502cfb80b67ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:10.987765 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1008/2044] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 13:07:10 2022] DELETE /identity/v3/users/912e26aec2724786816502cfb80b67ed => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:07:10.992500 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8188a7-fc45-4700-93fa-8332f847c52c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:10.993001 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8188a7-fc45-4700-93fa-8332f847c52c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:10.993244 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8188a7-fc45-4700-93fa-8332f847c52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:11.013216 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1d8188a7-fc45-4700-93fa-8332f847c52c None None] MFA Rules not processed for user `05701e65a5f34ab3837cd0f450bded1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:07:11.014431 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1d8188a7-fc45-4700-93fa-8332f847c52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:11.077408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1037/2045] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:07:11.087995 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:07:11.090105 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:11.148312 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b00a-c64e-41d4-8c24-5f860d80128c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:07:11.152103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc52fb-84a0-4288-8373-b6ea7ebe7f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:07:11.152714 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc52fb-84a0-4288-8373-b6ea7ebe7f3c service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:11.153770 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc52fb-84a0-4288-8373-b6ea7ebe7f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:11.154057 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc52fb-84a0-4288-8373-b6ea7ebe7f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:11.163370 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc52fb-84a0-4288-8373-b6ea7ebe7f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:07:11.164807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc52fb-84a0-4288-8373-b6ea7ebe7f3c service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:07:11.174681 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1009/2046] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:07:11.608982 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8188a7-fc45-4700-93fa-8332f847c52c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:07:11.611320 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1d8188a7-fc45-4700-93fa-8332f847c52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:11.631178 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8188a7-fc45-4700-93fa-8332f847c52c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:07:11.636701 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05701e65a5f34ab3837cd0f450bded1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90479a5430a046cea76f80fced5060d9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:07:11.637408 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:11.637638 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:11.637927 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] PATH_INFO: `/v3/projects/457ba94b915d49b9b922982cfeb4535e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:11.639115 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] RBAC: Authorizing `identity:delete_project(project_id=457ba94b915d49b9b922982cfeb4535e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:07:11.641308 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:07:11.759259 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457ba94b915d49b9b922982cfeb4535e', 'request_id': 'req-fb91ab98-3328-41c5-8244-5fe5e9c1795a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:11.820461 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457ba94b915d49b9b922982cfeb4535e', 'request_id': 'req-fb91ab98-3328-41c5-8244-5fe5e9c1795a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:11.827627 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457ba94b915d49b9b922982cfeb4535e', 'request_id': 'req-fb91ab98-3328-41c5-8244-5fe5e9c1795a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:11.838610 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] The token cache is being invalidate because project 457ba94b915d49b9b922982cfeb4535e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:07:11.839467 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:11.839730 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fb91ab98-3328-41c5-8244-5fe5e9c1795a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:07:11.841365 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1038/2047] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 13:07:11 2022] DELETE /identity/v3/projects/457ba94b915d49b9b922982cfeb4535e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:07:11.909332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc52fb-84a0-4288-8373-b6ea7ebe7f3c service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:07:11.910947 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77dc52fb-84a0-4288-8373-b6ea7ebe7f3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:11.923930 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc52fb-84a0-4288-8373-b6ea7ebe7f3c service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:07:11.943676 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:07:11.979015 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:07:11.980049 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:11.980327 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:11.980792 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:11.982652 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:12.053944 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:07:12.055384 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:07:12.064940 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1010/2048] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:07:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:07:21.835509 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:07:22.060529 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=40 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:07:24.922179 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79783364-f7f5-43d9-9a7c-a59456d3f849 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:24.922699 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79783364-f7f5-43d9-9a7c-a59456d3f849 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:24.923036 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79783364-f7f5-43d9-9a7c-a59456d3f849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:24.943561 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-79783364-f7f5-43d9-9a7c-a59456d3f849 None None] MFA Rules not processed for user `16c94590fafa48f58fd94526a913dd66`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:07:24.944867 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-79783364-f7f5-43d9-9a7c-a59456d3f849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:25.003587 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1039/2049] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 13:07:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:07:25.052021 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:07:25.053511 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:25.125007 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929fc89-b0b1-4991-8c92-eed58ba973a0 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:07:25.130145 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52b233-a740-4955-9923-63c318a8af84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b849f213606a4686bfb140fec34d512f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:07:25.130784 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52b233-a740-4955-9923-63c318a8af84 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:25.131069 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52b233-a740-4955-9923-63c318a8af84 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:25.131348 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52b233-a740-4955-9923-63c318a8af84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:25.140532 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52b233-a740-4955-9923-63c318a8af84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:07:25.142182 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52b233-a740-4955-9923-63c318a8af84 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:07:25.150695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1011/2050] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 13:07:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:07:25.373226 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a651a-25da-489f-bb6b-505a3780b7a9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:25.373526 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a651a-25da-489f-bb6b-505a3780b7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:25.373810 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a651a-25da-489f-bb6b-505a3780b7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:25.403900 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-739a651a-25da-489f-bb6b-505a3780b7a9 None None] MFA Rules not processed for user `f2b548d559b54575afd98739340ccf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:07:25.408979 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-739a651a-25da-489f-bb6b-505a3780b7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:25.475266 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1040/2051] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:07:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:07:35.147822 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=32 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:07:35.466381 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=50 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:07:44.691286 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886da3d-7ea5-4f10-9407-781c051ed024 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:44.691866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886da3d-7ea5-4f10-9407-781c051ed024 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:44.691866 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886da3d-7ea5-4f10-9407-781c051ed024 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:44.693411 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1012/2052] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 13:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:07:44.699136 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fc523-9dbd-4aad-b997-227b08ce4087 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:44.699648 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fc523-9dbd-4aad-b997-227b08ce4087 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:44.699745 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fc523-9dbd-4aad-b997-227b08ce4087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:44.719735 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-989fc523-9dbd-4aad-b997-227b08ce4087 None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:07:44.720988 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-989fc523-9dbd-4aad-b997-227b08ce4087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:44.795121 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1041/2053] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 13:07:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:07:44.807090 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886da3d-7ea5-4f10-9407-781c051ed024 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:07:44.809447 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2886da3d-7ea5-4f10-9407-781c051ed024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:44.884398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886da3d-7ea5-4f10-9407-781c051ed024 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:07:44.888271 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedab0c-0a76-4d7d-a307-397746836dc2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc4d28cded84afba73b053d18d4aa23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:07:44.888890 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedab0c-0a76-4d7d-a307-397746836dc2 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:44.889173 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedab0c-0a76-4d7d-a307-397746836dc2 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:44.889480 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedab0c-0a76-4d7d-a307-397746836dc2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:44.900851 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedab0c-0a76-4d7d-a307-397746836dc2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:07:44.903188 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedab0c-0a76-4d7d-a307-397746836dc2 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:07:44.914332 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1013/2054] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:07:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:07:45.130507 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918b111-9f40-4da2-bede-421dffa1d5a1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:45.130796 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918b111-9f40-4da2-bede-421dffa1d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:45.131070 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918b111-9f40-4da2-bede-421dffa1d5a1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:45.132470 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1042/2055] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 13:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:07:45.138398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918562b-81d3-495f-9f06-91654b212399 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:45.138699 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918562b-81d3-495f-9f06-91654b212399 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:45.138974 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918562b-81d3-495f-9f06-91654b212399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:45.158901 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c918562b-81d3-495f-9f06-91654b212399 None None] MFA Rules not processed for user `7a459d7bb6ea42ed93c5bd8eea8a37b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:07:45.160081 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c918562b-81d3-495f-9f06-91654b212399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:45.167114 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1014/2056] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 13:07:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:07:45.180508 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918b111-9f40-4da2-bede-421dffa1d5a1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:07:45.182346 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a918b111-9f40-4da2-bede-421dffa1d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:45.196326 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918b111-9f40-4da2-bede-421dffa1d5a1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:07:45.199891 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370c340-6b6e-4b9e-b92e-136e2f1ed85c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:07:45.201741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370c340-6b6e-4b9e-b92e-136e2f1ed85c service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:45.201741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370c340-6b6e-4b9e-b92e-136e2f1ed85c service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:45.201741 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370c340-6b6e-4b9e-b92e-136e2f1ed85c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:45.208299 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370c340-6b6e-4b9e-b92e-136e2f1ed85c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:07:45.209790 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370c340-6b6e-4b9e-b92e-136e2f1ed85c service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:07:45.217915 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1043/2057] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:07:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:07:45.299382 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c962b-d2e8-44ca-9aeb-c7773996d8c7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:45.299669 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c962b-d2e8-44ca-9aeb-c7773996d8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:45.299952 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c962b-d2e8-44ca-9aeb-c7773996d8c7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:45.301366 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1015/2058] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 13:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:07:45.307102 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9ce0d-a930-49df-b68f-81f59e6a4efd None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:45.307473 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9ce0d-a930-49df-b68f-81f59e6a4efd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:45.307819 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9ce0d-a930-49df-b68f-81f59e6a4efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:45.328732 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-11f9ce0d-a930-49df-b68f-81f59e6a4efd None None] MFA Rules not processed for user `44b2b449acaa45f099c8a56ecbf20ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:07:45.329989 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-11f9ce0d-a930-49df-b68f-81f59e6a4efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:45.339235 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1044/2059] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 13:07:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:07:45.350025 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c962b-d2e8-44ca-9aeb-c7773996d8c7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:07:45.361443 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c962b-d2e8-44ca-9aeb-c7773996d8c7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:07:45.365092 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc5b83-fff3-4132-b7f1-28733044b153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:07:45.365741 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc5b83-fff3-4132-b7f1-28733044b153 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:45.366017 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc5b83-fff3-4132-b7f1-28733044b153 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:45.366288 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc5b83-fff3-4132-b7f1-28733044b153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:45.376489 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc5b83-fff3-4132-b7f1-28733044b153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:07:45.377969 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc5b83-fff3-4132-b7f1-28733044b153 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:07:45.389510 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1016/2060] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:07:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:07:49.667454 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076786f-7261-4f15-a5f4-7eb54bbb799c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:49.667923 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076786f-7261-4f15-a5f4-7eb54bbb799c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:49.668017 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076786f-7261-4f15-a5f4-7eb54bbb799c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:49.688566 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e076786f-7261-4f15-a5f4-7eb54bbb799c None None] MFA Rules not processed for user `16c94590fafa48f58fd94526a913dd66`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:07:49.689789 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e076786f-7261-4f15-a5f4-7eb54bbb799c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:07:49.696694 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1045/2061] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 13:07:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:07:49.708649 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc5b83-fff3-4132-b7f1-28733044b153 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:07:49.719976 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc5b83-fff3-4132-b7f1-28733044b153 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:07:49.724895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de0acf-4930-4a77-8359-5006009ea0a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b849f213606a4686bfb140fec34d512f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:07:49.725530 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de0acf-4930-4a77-8359-5006009ea0a3 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:07:49.725814 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de0acf-4930-4a77-8359-5006009ea0a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:07:49.726091 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de0acf-4930-4a77-8359-5006009ea0a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:07:49.734481 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de0acf-4930-4a77-8359-5006009ea0a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:07:49.736097 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de0acf-4930-4a77-8359-5006009ea0a3 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:07:49.744055 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1017/2062] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 13:07:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:07:59.683271 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=43 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:07:59.740873 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=41 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:08:20.427416 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0acd11-1648-4560-8622-94015d10c434 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:08:20.427952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0acd11-1648-4560-8622-94015d10c434 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:08:20.427952 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0acd11-1648-4560-8622-94015d10c434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:08:20.447925 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7e0acd11-1648-4560-8622-94015d10c434 None None] MFA Rules not processed for user `16c94590fafa48f58fd94526a913dd66`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:08:20.449138 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7e0acd11-1648-4560-8622-94015d10c434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:08:20.456803 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1046/2063] 10.222.0.30 () {58 vars in 967 bytes} [Sat Oct 15 13:08:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:08:20.469793 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de0acf-4930-4a77-8359-5006009ea0a3 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:08:20.480842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de0acf-4930-4a77-8359-5006009ea0a3 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:08:20.486398 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86e2ba-8a25-4b33-b315-48eb92d58dc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b849f213606a4686bfb140fec34d512f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:08:20.487146 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86e2ba-8a25-4b33-b315-48eb92d58dc8 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:08:20.487430 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86e2ba-8a25-4b33-b315-48eb92d58dc8 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:08:20.487705 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86e2ba-8a25-4b33-b315-48eb92d58dc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:08:20.495604 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86e2ba-8a25-4b33-b315-48eb92d58dc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:08:20.497208 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86e2ba-8a25-4b33-b315-48eb92d58dc8 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:08:20.506138 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1018/2064] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 13:08:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:08:26.473756 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0acd11-1648-4560-8622-94015d10c434 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:08:26.483022 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0acd11-1648-4560-8622-94015d10c434 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:08:26.486924 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f100b48-2a92-4c0b-8e10-117b0cc62233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:08:26.487523 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f100b48-2a92-4c0b-8e10-117b0cc62233 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:08:26.487801 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f100b48-2a92-4c0b-8e10-117b0cc62233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:08:26.488073 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f100b48-2a92-4c0b-8e10-117b0cc62233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:08:26.489838 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8f100b48-2a92-4c0b-8e10-117b0cc62233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:08:26.502781 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f100b48-2a92-4c0b-8e10-117b0cc62233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:08:26.504207 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f100b48-2a92-4c0b-8e10-117b0cc62233 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:08:26.514932 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1047/2065] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:08:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:08:27.695894 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86e2ba-8a25-4b33-b315-48eb92d58dc8 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:08:27.697404 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7c86e2ba-8a25-4b33-b315-48eb92d58dc8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:08:27.709122 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86e2ba-8a25-4b33-b315-48eb92d58dc8 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:08:27.712625 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:08:27.713256 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:08:27.713535 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:08:27.713805 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:08:27.715472 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:08:27.728809 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:08:27.730254 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:08:27.741807 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1019/2066] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:08:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:08:36.510292 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=9 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:08:37.737018 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=12 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:08:39.647425 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd5883-8fdb-4aa8-92c5-f5fd22478c8a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:08:39.647954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd5883-8fdb-4aa8-92c5-f5fd22478c8a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:08:39.647954 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd5883-8fdb-4aa8-92c5-f5fd22478c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:08:39.667108 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c7cd5883-8fdb-4aa8-92c5-f5fd22478c8a None None] MFA Rules not processed for user `c36b6b9fbf194346bfc0a7e277c72eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:08:39.668315 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c7cd5883-8fdb-4aa8-92c5-f5fd22478c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:08:39.722704 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1048/2067] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 13:08:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:08:45.973465 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:08:45.974895 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:08:45.988646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ccc1a-a624-4332-bc95-096559c63efd service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:08:45.994231 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4df40-a7f2-44f8-9edf-2cbc61d7b5f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:08:45.994842 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4df40-a7f2-44f8-9edf-2cbc61d7b5f2 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:08:45.995120 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4df40-a7f2-44f8-9edf-2cbc61d7b5f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:08:45.995395 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4df40-a7f2-44f8-9edf-2cbc61d7b5f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:08:45.997217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-28e4df40-a7f2-44f8-9edf-2cbc61d7b5f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:08:46.007646 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4df40-a7f2-44f8-9edf-2cbc61d7b5f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:08:46.012018 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4df40-a7f2-44f8-9edf-2cbc61d7b5f2 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:08:46.018206 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1020/2068] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:08:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:08:48.810040 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd5883-8fdb-4aa8-92c5-f5fd22478c8a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:08:48.811372 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c7cd5883-8fdb-4aa8-92c5-f5fd22478c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:08:48.827233 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd5883-8fdb-4aa8-92c5-f5fd22478c8a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:08:48.833127 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b4ce9-5ccf-47ad-9e3a-13a4f9f21683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b2b449acaa45f099c8a56ecbf20ca6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:08:48.833749 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b4ce9-5ccf-47ad-9e3a-13a4f9f21683 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:08:48.834026 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b4ce9-5ccf-47ad-9e3a-13a4f9f21683 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:08:48.834297 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b4ce9-5ccf-47ad-9e3a-13a4f9f21683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:08:48.835930 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0d4b4ce9-5ccf-47ad-9e3a-13a4f9f21683 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:08:48.892160 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b4ce9-5ccf-47ad-9e3a-13a4f9f21683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:08:48.893772 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b4ce9-5ccf-47ad-9e3a-13a4f9f21683 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:08:48.903073 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1049/2069] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:08:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:08:56.014035 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:08:58.898947 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=53 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:09:39.841695 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4df40-a7f2-44f8-9edf-2cbc61d7b5f2 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:09:39.850869 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4df40-a7f2-44f8-9edf-2cbc61d7b5f2 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:09:39.854434 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c7b0c-bd8b-4a01-87b9-99733422d193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a459d7bb6ea42ed93c5bd8eea8a37b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:09:39.855026 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c7b0c-bd8b-4a01-87b9-99733422d193 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:09:39.855304 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c7b0c-bd8b-4a01-87b9-99733422d193 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:09:39.855573 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c7b0c-bd8b-4a01-87b9-99733422d193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:09:39.857283 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-907c7b0c-bd8b-4a01-87b9-99733422d193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:09:39.867103 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c7b0c-bd8b-4a01-87b9-99733422d193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:09:39.868513 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c7b0c-bd8b-4a01-87b9-99733422d193 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:09:39.876259 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1021/2070] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:09:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:09:49.872843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:12:42.723571 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81dace-47e4-4c2f-8168-12baa011a314 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:12:42.724018 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81dace-47e4-4c2f-8168-12baa011a314 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:12:42.724123 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81dace-47e4-4c2f-8168-12baa011a314 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:12:42.725477 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1050/2071] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 13:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:12:42.731217 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8179f-0825-4a2b-ba1e-232c56232149 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:12:42.731657 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8179f-0825-4a2b-ba1e-232c56232149 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:12:42.731843 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8179f-0825-4a2b-ba1e-232c56232149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:12:42.811357 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-57a8179f-0825-4a2b-ba1e-232c56232149 None None] MFA Rules not processed for user `18df1f401bb2494299eda9582c85a04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:12:42.812482 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-57a8179f-0825-4a2b-ba1e-232c56232149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:12:42.923038 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1022/2072] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 13:12:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:12:42.931524 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81dace-47e4-4c2f-8168-12baa011a314 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:12:42.944577 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81dace-47e4-4c2f-8168-12baa011a314 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:12:42.949995 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974ec14-02b3-45b6-95d7-ec8b22761131 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18df1f401bb2494299eda9582c85a04f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c6d59e727614128a4304872ad0867cd', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:12:42.950700 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974ec14-02b3-45b6-95d7-ec8b22761131 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:12:42.950987 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974ec14-02b3-45b6-95d7-ec8b22761131 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:12:42.951264 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974ec14-02b3-45b6-95d7-ec8b22761131 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:12:42.952993 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b974ec14-02b3-45b6-95d7-ec8b22761131 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:12:42.964406 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974ec14-02b3-45b6-95d7-ec8b22761131 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:12:42.966023 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974ec14-02b3-45b6-95d7-ec8b22761131 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:12:42.975600 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1051/2073] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 13:12:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:12:52.921314 dv-cinder-861491-2-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=74 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:12:52.971483 dv-cinder-861491-2-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}